Methods and devices for controlling a shapeable medical device

ABSTRACT

Systems and methods are described herein that improve control of a shapeable or steerable instrument using shape data. Additional methods include preparing a robotic medical system for use with a shapeable instrument and controlling advancement of a shapeable medical device within an anatomic path. Also described herein are methods for altering a data model of an anatomical region.

FIELD OF THE INVENTION

The invention relates generally to medical instruments, such as elongatesteerable instruments for minimally-invasive intervention or diagnosis,and more particularly to a method, system, and apparatus for sensing Ormeasuring the shape or position and shape of one or more parts of ashapeable elongate medical instrument.

BACKGROUND

Currently known minimally invasive procedures for diagnosis andtreatment of medical conditions use shapeable instruments, such assteerable devices, flexible catheters or more rigid arms or shafts, toapproach and address various tissue structures within the body.Hereafter, such devices are referred to as “shapeable” instruments. Sucha term can include steerable devices, flexible devices, devices havingone or more pre-determined shapes (such as an articulating device thatlocks into a particular shape). For various reasons, it is highlyvaluable to be able to determine the 3-dimensional spatial position ofportions of such shapeable instruments relative to other structures,such as the operating table, other instruments, or pertinent anatomicaltissue structures. Such information can be used for a variety ofreasons, including, but not limited to: improve device control; toimprove mapping of the region; to adapt control system parameters(whether kinematic and/or solid mechanic parameters); to estimate, planand/or control reaction forces of the device upon the anatomy; and/or toeven monitor the system characteristics for determination of mechanicalproblems. Alternatively, or in combination, shape information can beuseful to simply visualize the tool with respect to the anatomy or otherregions whether real or virtual

Conventional systems can be improved by incorporating shape informationinto the control of the medical device. To better understand suchimprovements a discussion of the concept of shape might be useful. Mostgenerally, shape can include geometric information about an objectwithout information of location, scale or rotation. While the discussionfocuses on the use of robotics to control a shapeable device, theconcepts disclosed herein can be applied to any robotic, automated, ormachine assisted control of a medical device.

Shape can be important for improved control of shapeable devices. In thefield of discrete robotics, joint positions are used extensively todescribe relative positions of connected articulating members. In thecase of a shapeable instrument being advanced by a robotic or othersystem, there are effectively infinite joints with multiple degrees offreedom. Instead of just knowing the scalar or vector configuration of ajoint, the shape of a shapeable section is needed and must be eitherinferred or measured.

A machine that controls a shapeable medical device carries force andflexure continuously through sections with some degree of smoothness inone or more path derivatives. The shape of these sections can providevaluable information. However, purely defined shape excludes location,rotation and scaling of a body. Shape as described hereafter generallyincludes shape with scale. Thus with the shape, the relative positionand orientation of any two points on the shape are known. For example,as shown in FIG. 1A, if the shape S of shapeable instrument 1 is knownand coordinates are known or assigned for q, relative coordinates andorientation may also be assigned for q′ in the reference frame of q. Inother words, knowing the shape allows for all point in the body to bedefined relative to a reference frame in the body. To reiterate, thereference frame or point is on the shapeable instrument rather than onthe actual robot or controlling device.

Without shape measurement, other information must be used to control ashapeable device. However, such control is subject to error by multiplesources. FIG. 1B shows an example of an overview block diagram of abasic topology used for controlling devices without shape feedback. Theleft side of the diagram (or the Desired/Master side) describes thedesired behavior of the catheter (sometimes also referred to as virtualside). The right side of the diagram (referred to as the real, actual,or slave side) describes the behavior of the actual physical catheter.Both sides provide a description of the catheter into at least threelevels: tip position (task space), catheter configuration (configurationor joint space), and tendon displacements (actuator space).

FIG. 1B also illustrates a typical control flow for a basic cathetercontrol. The operator enters a command to designate a desired tipposition for the device via some input mechanism (a master input device,computer software, or other “user interface, etc.). Next, one or moreinverse kinematic algorithms compute a desired catheter configuration inorder to achieve the commanded tip position. The inverse kinematicalgorithm can be varied depending on the construction of the shapeabledevice. The desired catheter configuration is then fed to one or morecatheter mechanics algorithm to compute the positioning elementdisplacements necessary to achieve the desired catheter configuration.These positioning element commands are then provided to the robotscontrol algorithms (or in some cases actuators in the robot thatinterface with positioning elements in the shapeable element).

Based upon the applied positioning element displacements, the actual(physical) catheter mechanics including any constraints and obstructionsacting on the catheter determine the real configuration or shape thatthe shapeable device achieves. This is illustrated on the right(slave/actual) side of FIG. 1B. This real catheter configuration/shapedetermines the real catheter tip position. These kinematic relationshipsof the physical device are represented in the figure with a forwardkinematics block (50). Assuming that the operator is observing thecatheter tip through some sort of visualization (fluoro, endoscopy,etc), the operator can then use this visual feedback to make correctionsto the commanded tip position. However, this form of feedback is basedon the human operator's perception and skill, which vary betweenindividuals not to mention that an individual's perception of thefeedback can vary during a procedure or over a number of procedures.

To generate the control inputs, the system must calculate inversekinematics and translate to configuration space. These mathematicaloperations are essentially inverted by the physical system in actuatingthe device, subject to disturbances such as interference with theenvironment.

In many conventional systems, the catheter (or other shapeableinstrument) is controlled in an open-loop manner as shown in FIG. 1C. Inthis type of open loop control model, the shape configuration commandcomes in to the beam mechanics, is translated to beam moments andforces, then is translated to tendon tensions given the actuatorgeometry, and finally into tendon displacement given the entire deformedgeometry. However, there are numerous reasons why the assumed motion ofthe catheter will not match the actual motion of the catheter, oneimportant factor is the presence of unanticipated or unmodeledconstraints imposed by the patient's anatomy.

Clearly, the presence of unanticipated or unmodeled portions of theanatomy affects the behavior and therefore kinematics of the shapeableinstrument. This affect will often alter any mapping betweenconfiguration or shape and task space or endpoint for the instrument.FIG. 1D shows a basic example of this situation. When a section of ashapeable instrument articulates without encountering an obstruction(from “a” to “b”), the tip of the instrument (1) moves along an arc thatis now oriented largely vertically. When the instrument (1) encountersan environmental constraint (49), the constraint (49) limits themovement of the tip of the instrument (1) in a tighter arc. In mostcases, the controller that issues signals to direct the instrument (1)does not account for the presence of this constraint (49), so anyinverse kinematic analysis assumes that the instrument (1) is in theshape depicted in B while in reality is in the altered shape depicted in“c”.

Accordingly, a control system that directs shapeable instruments cancommand joint configurations that can achieve a desired tip position.However, the presence of modeling inaccuracies and environmentinteraction causes a differential between the actual position from thatintended. A simple tip position can quantify this error, but addressingthe source of the error requires the additional information regardingthe shapeable instrument. Data defining the actual or real shape of theinstrument can provide much of this information.

Conventional technologies such as electromagnetic position sensors,available from providers such as the Biosense Webster division ofJohnson & Johnson, Inc., can be utilized to measure 3-dimensionalspatial position but may be limited in utility for elongate medicalinstrument applications due to hardware geometric constraints,electromagnetivity issues, etc.

It is well known that by applying the Bragg equation(wavelength=2*d*sin(theta)) to detect wavelength changes in reflectedlight, elongation in a diffraction grating pattern positionedlongitudinally along a fiber or other elongate structure may bedetermined. Further, with knowledge of thermal expansion properties offibers or other structures which carry a diffraction grating pattern,temperature readings at the site of the diffraction grating may becalculated.

“Fiberoptic Bragg grating” (“FBG”) sensors or components thereof,available from suppliers such as Luna Innovations, Inc., of Blacksburg,Va., Micron Optics, Inc., of Atlanta, Ga., LxSix Photonics, Inc., ofQuebec, Canada, and Ibsen Photonics A/S, of Denmark, have been used invarious applications to measure strain in structures such as highwaybridges and aircraft wings, and temperatures in structures such assupply cabinets.

The use of such technology in shapeable instruments is disclosed incommonly assigned U.S. patent application Ser. Nos. 11/690,116;11/176,598; 12/012,795; 12/106,254; and 12/507,727. Such technology isalso described in U.S. Provisional application Nos. 60/785,001;60/788,176; 60/678,097; 60/677,580; 60/600,869; 60/553,029; 60/550,961;60/644,505. The entirety of each of the above applications isincorporated by reference herein. Related disclosures of systems andmethods for controlling a shapeable instrument can be found U.S. Ser.No. 12/822,876, filed Jun. 24, 2010, the entirety of which isincorporated by reference.

There remains a need to apply the information gained by the spatialinformation or shape and applying this information to produce improveddevice control or improved modeling when directing a robotic or similardevice. There also remains a need to apply such controls to medicalprocedures and equipment.

SUMMARY OF THE INVENTION

The systems, methods, and devices described herein include a roboticmedical system for controlling a shapeable instrument within ananatomical region. The systems, methods, and devices described hereinincorporate shape measurement and apply the measured information asfeedback for controlling of the shapeable member or for performing othertasks related to controlling the instrument (e.g., improving a map or amodel of the anatomy or region).

The shapeable medical instruments, in most variations described herein,include any steerable devices, flexible catheters or more rigid arms orshafts whether such devices are used to access a region for advancementof a treatment device, or any actual shapeable treatment device. Ashapeable device as used herein include includes flexible, steerable, orotherwise positionable devices that are advanced to various tissuestructures within the body. Such devices can assume a shapedconfiguration via manipulation or steering. Moreover, shapeable devicesinclude those flexible devices that conform to anatomic or otherobstructions. In many variations, shapeable instruments include aworking end and one or more positioning elements that move the shapeableinstrument. In one example, the positioning elements comprise controlelements such as tendons wires, or other mechanical structures that aremoved by one or more actuators to affect a shape or reposition theshapeable instrument. Unless specifically used to indicate a particulardevice, the term catheter is one example of a shapeable instrument.

In a first variation, the robotic medical system comprises a medicalsystem for controlling a shapeable instrument within an anatomicalregion, where the shapeable instrument includes at least a workingsection and one or more positioning elements that move the shapeableinstrument.

One variation of the system includes a controller including a masterinput device, where the controller generates a′ position control signalin response to the master input device to position the working sectionat a desired position; one or more actuators operatively coupleable tothe one or more positioning elements, where the actuators manipulate thepositioning elements based on the position control signal to drive atleast a first portion of the shapeable instrument to position theworking section toward the desired position; a localization systemconfigured to obtain a plurality of localized shape data from the firstportion of the shapeable instrument; and where the controller generatesa signal based upon a differential between the localized shape data anda desired configuration of the first portion of the shapeableinstrument. The desired configuration of the first portion can include adesired position of the first portion or the desired position of theworking section. Alternatively, or in combination, the desiredconfiguration of the first portion comprises a desired shape of thefirst portion.

The localization system can determine a position of the working sectionfrom the plurality of localized shape data. In another variation, thedesired configuration of the first portion comprises a desired positionof the first portion and where controller generates the signal basedupon the differential between the position of the working section andthe desired position of the working section. The controller of therobotic medical system can be configured to derive a position of theworking section from a kinematic model of the shapeable instrument.

A variation of the robotic medical system includes a localization systemthat determines a shape of the first portion of the shapeable instrumentfrom the plurality of localized shape data. The desired configuration ofthe first portion can comprises a desired shape of the first portion andwhere controller generates the signal based upon the differential,between the shape of the first portion and the desired shape of thefirst portion. In another variation, the localization system alsodetermines a position of the working section from the plurality oflocalized shape data, and where the desired configuration of the firstportion also includes a desired position of the first portion.

In another variation, the controller generates the signal also basedupon the differential between a desired position of the first portionand the position of the first portion.

The robotic medical system can also include a controller that isconfigured to feed the signal to the actuators such that the actuatorsmanipulate one or more of the positioning elements using the signal toposition the working section or the first portion of the shapeableinstrument.

In one variation, the localization system comprises a fiber opticlocalization system configured to supply the plurality of localizationdata. Furthermore, the shapeable instrument can include at least oneoptic fiber and where the localization system is configured to measure aplurality of data of Rayleigh scatter of the optic fiber. The Rayleighscatter data can be used to supplement or supply the localization data.

The localization system can comprises a system selected from the groupconsisting of a plurality of positioning sensors, a vision system, aplurality of strain sensors. In another variation, the localizationsystem can comprise an electromagnetic localization system and where theshapeable instrument includes at least one electromagnetic coil. In yetanother variation, the localization system can comprise an impedancebased localization system and where the shapeable instrument includes atleast one sensor, where the system further includes at least oneelectrode where the impedance based localization system determines avoltage gradient between the sensor and the electrode. However, anynumber of localization systems can be employed with the robotic medicalsystem as described herein.

The robotic medical system described herein can also include a controlconfigured to generate the position control feed signal using an inversekinematic model of the shapeable instrument. The controller can generatethe position control signal to maximize a probability of achieving aprescribed shape or position by optimizing a cost function subject to aset of constraints based upon a model and a measurement estimate. Forexample, the model can be selected from the group consisting of akinematic and solid mechanic model and the measurement projection can beselected from the group consisting of a shape, a strain, and aprojection (e.g., a fluoroscopic projection, etc.).

In one variation, the controller can use the signal to alter at leastone parameter of the inverse kinematic model of the shapeable instrumentto produce an improved inverse kinematic model of the shapeableinstrument. Furthermore, the controller can modify the position controlsignal using the improved kinematic model.

The robotic medical system described herein can also be configured suchthat the actuators alter a force applied to one or more of thepositioning elements based on the signal to reposition the workingsection or the first portion of the shapeable instrument.

In yet another variation, the robotic medical system can be furtherconfigured to measure an axial deformation of the shapeable member, andwhere the controller further generates the signal based on the axialdeformation of the shapeable member.

In variations of the robotic medical system the controller can beconfigured to determine an applied force on the first portion of theshapeable instrument using a shape of the first portion of the shapeableinstrument, the position control signal and at least one characteristicof the shapeable instrument. The controller can further use one or moreactuators to reposition the portion of the shapeable instrument toreduce the applied force.

In another variation, when the robotic medical system generates thesignal, the controller can trigger an operator alert signal alarm to themaster input device. The operator signal can cause a haptic effect onthe master input device for feedback. In some variations, the controllertriggers the operator alert signal only if the signal is greater than apre-determined level. Any number of safety measures can be employed whenthe controller triggers the operator alert signal. For example, thecontroller can be configured to stop movement of the shapeableinstrument; the controller can be configured to reverse movement of theshapeable instrument; and/or the controller can be configured toincrease a force required to operate the master input device.

The controller of the robotic medical system described herein can beconfigured to further determine a calculated curvature from the realshape and compares the calculated curvature to a pre-determinedcurvature to assess a fracture of the shapeable element.

The present disclosure also includes methods for controlling a shapeableinstrument within an anatomical region using a robotic medical system.For example, the method can include operatively coupling one or moreactuators to one or more positioning elements of a shapeable instrumentwhere the one or more positioning elements are adapted to move theshapeable instrument and where the actuators manipulate the positioningelements; advancing the shapeable instrument to the anatomical region,where the shapeable instrument includes a working section; generating aposition control signal in response to position the working section at adesired position; obtaining a plurality of localized shape data of afirst portion of the shapeable instrument using a localization system;and controlling the actuators using the position control signal tomanipulate the positioning control elements to drive at least the firstportion of the shapeable instrument to position the working sectiontoward the desired position, where the controller generates a signalbased upon a differential between the localized shape data and a desiredconfiguration of the first portion of the shapeable instrument.

The methods described herein can also permit tracking of a devicethrough an anatomic path using shape information of the device. Forinstance, one method includes controlling advancement of a shapeablemedical device within an anatomic path. Such variation includesidentifying a reference shape of one or more portions of the shapeablemedical device; advancing the shapeable medical device along theanatomic path; obtaining a plurality of localization data to determine areal shape of at least the one or more portions of the shapeableinstrument when advanced along the anatomic path; and monitoringadvancement of the shapeable medical device by determining adifferential between the real shape and the reference shape of the oneor more portions.

In addition, the method of controlling advancement of the medical devicecan further comprise controlling advancement of the shapeable medicaldevice if the differential between the real shape and the referenceshape of the one or more portions is greater than a threshold value.

In one variation controlling the advancement of the shapeable medicaldevice comprises reversing the shapeable medical device along theanatomic path until the differential between the real shape and thereference shape decreases. In another variation, controlling theadvancement of the shapeable medical device comprises slowingadvancement of the shapeable medical device along the anatomic pathuntil the differential between the real shape and the reference shapedecreases. Another variation controlling the advancement of theshapeable medical device comprises advancing a guide device from theshapeable medical device within the anatomic path and subsequentlyadvancing the shapeable medical device along the guide track. Inadditional variations controlling the advancement of the shapeablemedical device comprises stopping the shapeable medical device andwithdrawing a proximal end of the shapeable medical device until thedifferential between the real shape and the reference shape decreases.

The disclosure also includes method for reduced model control. One suchmethod includes altering a data model of an anatomical region. Forexample, such a method can include advancing a shapeable instrumentrelative to the anatomical region, the shapeable instrument comprisingone or more positioning elements that alter a shape of a first portionof the shapeable instrument obtaining a plurality of localization datato determine a real shape of the first portion of the shapeableinstrument; correlating the real shape of the first portion of theshapeable instrument against a desired shape of the first portion todetermine a data model of an anatomic feature affecting the real shapeof at least the first portion of the shapeable instrument; and updatingthe data model with the data model of the anatomic feature.

The method described above can further comprise measuring at least oneforce on at least one positioning element and where correlating the realshape of the first portion of the shapeable instrument includesassessing the force on the at least one positioning element to determinethe data model of the anatomic feature affecting the real shape of atleast the first portion of the shapeable instrument. The methods canfurther include cycling movement of the shapeable instrument byadvancing and retracting the shapeable instrument, and where obtainingthe localization data occurs after advancing the shapeable instrument.

The methods described herein can repositioning the shapeable instrumentto maintain a historical database of real shapes. where the historicaldatabase comprises a plurality of active spaces through which theshapeable instrument moved and a plurality of void spaces through whichthe shapeable instrument did not move, and determining a location of anatomic feature using the plurality of void spaces.

The present disclosure also include methods of preparing a roboticmedical system for use with a shapeable instrument, where the shapeableinstrument includes a working end and one or more positioning elementsthat move the shapeable instrument. A variation of this method includesobtaining a plurality of localization data to determine a real shape ofat least the first portion of the shapeable instrument; pretensioningthe shapeable instrument by incrementally actuating at least one of theactuators to determine a zero displacement point of the actuator afterwhich the shapeable instrument moves from the real shape; providing thezero displacement point to a controller including a master input device,where the controller adds the first displacement point to at least oneactuation command where the actuation command manipulates one or more ofthe positioning elements to reposition the working end or the firstportion of the shapeable instrument and where the first displacementpoint compensates for slack in the shapeable element.

In one example, a shapeable instrument comprises an elongate instrumentbody; an optical fiber coupled in a constrained manner to the elongateinstrument body, the optical fiber is in communication with one or moreoptical gratings; and a detector operably coupled to a proximal end ofthe optical fiber and configured to detect respective light signalsreflected by the one or more optical gratings. The system furtherincludes a controller operatively coupled to the detector, wherein thecontroller is configured to determine a geometric configuration of atleast a portion of the shapeable instrument based on a spectral analysisof the detected reflected portions of the light signals. Variations ofthe devices, systems and methods described herein can employ Bragg Fibergratings as mentioned above. However, additional variations of thedevices, systems and method contained in this disclosure can employ anynumber of optical gratings.

The systems, methods, and devices described herein can also employalternate means to obtain information regarding shape of the device. Forexample, such alternate means includes, but is not limited topositioning sensors, a vision system, a plurality of strain sensors.

By way of non-limiting example, a shapeable instrument can be robotically controlled, or manually controlled with automated assistance. Insome variations, the shapeable instrument includes a reference reflectorcoupled to the optical fiber in an operable relationship with the one ormore optical gratings. In yet additional embodiments, the detectorcomprises a frequency domain reflectometer. The optical fiber caninclude multiple fiber cores, each core including one or more opticalgratings. The optical fiber (or each fiber core of a multi-core opticalfiber) can optionally comprise a plurality of paced apart opticalgratings.

In another variation, a localization system as described herein can usemeasurement of Rayleigh scatter in the optical fiber. Measurement ofRayleigh scatter can be used to measure strain in the fiber. Suchinformation can be used as an alternate mode of obtaining shape data.Alternatively, Rayleigh scatter can be combined with other localizationsystems to supplement or improve the localized shape data.

When single mode optical fiber is drawn there can be slightimperfections that result in index of refraction variations along thefiber core. These variations result in a small amount of backscatterthat is called Rayleigh scatter. Changes in strain or temperature of theoptical fiber cause changes to the effective length of the opticalfiber. This change in the effective length results in variation orchange of the spatial position of the Rayleigh scatter points. Crosscorrelation techniques can measure this change in the Rayleighscattering and can extract information regarding the strain. Thesetechniques can include using optical frequency domain reflectometertechniques in a manner that is very similar to that associated with lowreflectivity fiber gratings. A more complete discussion of these methodscan be found in M. Froggatt and J. Moore, “High-spatial-resolutiondistributed strain measurement in optical fiber with Rayleigh scatter”,Applied Optics, Vol. 37, p. 1735, 1998 the entirety of which isincorporated by reference herein.

Methods and devices for calculating birefringence in an optical fiberbased on Rayleigh scatter as well as apparatus and methods for measuringstrain in an optical fiber using the spectral shift of Rayleigh scattercan be found in PCT Publication No. WO2006099056 tiled on Mar. 9, 2006and U.S. Pat. No. 6,545,760 filed on Mar. 24, 2000 both of which areincorporated by reference herein. Birefringence can be used to measureaxial strain and/or temperature in a waveguide. Using Rayleigh scatterto determine birefringence rather than Bragg gratings offers severaladvantages. First, the cost of using Rayleigh scatter measurement isless than when using Bragg gratings. Rayleigh scatter measurementpermits birefringence measurements at every location in the fiber, notjust at predetermined locations. Since Bragg gratings require insertionat specific measurement points along a fiber, measurement of Rayleighscatter allows for many more measurement points. Also, the process ofphysically “writing” a Bragg grating into an optical fiber can be timeconsuming as well as compromises the strength and integrity of thefiber. Such drawbacks do not occur when using Rayleigh scattermeasurement.

In various embodiments, the optical fiber may be substantiallyencapsulated in a wall of the elongate instrument body. Alternatively,the elongate instrument body may define an interior lumen, wherein theoptical fiber is disposed in the lumen. Further alternatively, theoptical fiber may be disposed in an embedded lumen in a wall of theelongate instrument body.

In various embodiments, the elongate instrument body has a neutral axisof bending, and the optical fiber is coupled to the elongate instrumentbody so as to be substantially aligned with the neutral axis of bendingwhen the elongate instrument body is in a substantially unbentconfiguration, and to move relative to the neutral axis of bending asthe elongate instrument body undergoes bending. In other embodiments,the optical fiber is coupled to the elongate instrument body so as to besubstantially aligned with the neutral axis of bending regardless ofbending of the elongate instrument body. In still further embodiments,the optical fiber is coupled to the elongate instrument body so as toremain substantially parallel to, but not aligned with, the neutral axisof bending regardless of bending of the elongate instrument body.

Shape feedback can be used directly along with system models in bothcontrol for task-space (e.g., the distal end) and/or configuration-space(the elongate portion). The configuration can be extended over time toplan for the environment, for example to track the shape inside avessel. On the other hand, a control architecture uses less devicemodels instead relying on the information rich shape feedback.

Shape feedback can also be used in device kinematics. Shape provides ameasurement of the real kinematics of a device. Kinematic parameters maybe estimated using the shape measurement. Extending that concept, shapemeasurement can be used to adapt the kinematic model by several methodsas described herein. Moreover, a real measured shape may be displayed toa system operator in addition to or in lieu of an idealized virtualshape.

Moving beyond the geometry of the device, the physical properties of thedevice materials, its solid mechanics, make up a fourth area. Addressinga specific challenge for elongate flexible devices, shape can be used tomeasure axial deformation. Shape can also be used to pretensionactuating tendons or control elements. More generally, real device shapemay be compared with model expectation to adapt real model parameters orestimate a state of health based on degradation of material properties.

Shape data can further assist in estimation and control of reactionforce between the device and environment. Deflection of measured shapefrom the predicted free shape belies application of external forces.Estimates of these forces may be used to navigate the environment or ifan environment model is available, plan a navigation path.

In another variation, use of data from shape feedback can be used todetect mechanical failures of the shapeable instrument. Such feedbackallows a mechanism for detecting, displaying, and handling mechanicalfractures. Basic diagnosis is extended with an active secondarydiagnostic to test potential fractures, redundant sensors formodel-based diagnosis and shape sensor diagnostics.

Other and further embodiments, objects and advantages of the inventionwill become apparent from the following detailed description when readin view of the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a general diagram to demonstrate the ability to determinecoordinates along a known shape.

FIG. 1B shows an example of an overview block diagram of a basictopology used for controlling devices without shape feedback.

FIG. 1C illustrates a conventional open loop control model.

FIG. 1D shows an example of a shapeable instrument articulating in freespace and when engaging an environmental constraint.

FIG. 2A illustrates an example of an elongate instrument such as aconventional manually operated catheter.

FIG. 2B illustrates another example of an elongate instrument such as arobotic ally-driven steerable catheter.

FIGS. 3A-3C illustrate implementations of an optical fiber with variousoptical gratings to an elongate instrument such as arobotically-steerable catheter.

FIGS. 4A-4D illustrate implementations of an optical fiber with agrating to an elongate instrument such as a robotically-steerablecatheter.

FIGS. 5A-5D illustrate implementation of an optical fiber with a gratingto an elongate instrument as a robotically-steerable catheter.

FIG. 6 illustrates a cross sectional view of an elongate instrument suchas a catheter including an optical fiber with optical gratings.

FIG. 7 illustrates a cross sectional view of an elongate instrument suchas a catheter including a multi-fiber optical grating configuration.

FIG. 8 illustrates a cross sectional view of an elongate instrument suchas a catheter including a multi-fiber grating configuration.

FIGS. 9A-9B illustrate top and cross sectional views of an elongateinstrument such as a catheter having a multi-fiber structure withoptical gratings.

FIGS. 10A-10B illustrate top and cross sectional views of an elongateinstrument such as a catheter having a multi-fiber structure withoptical gratings.

FIGS. 11A-11B illustrate top and cross sectional views of an elongateinstrument such as a catheter having a multi-fiber structure withoptical gratings.

FIGS. 12A-12H illustrate cross sectional views of elongate instrumentswith various fiber positions and configurations.

FIG. 13 illustrates an optical fiber sensing system with opticalgratings.

FIGS. 14A-14B illustrates an optical fiber sensing system with opticalgratings.

FIGS. 15A-15B illustrate optical fiber sensing system configurationswith optical gratings.

FIGS. 16A-16D illustrates integration of an optical fiber sensing systemto a robotically-controlled guide catheter configuration.

FIGS. 17A-F, 17G-1, and 17G-2 illustrate integration of an optical fibersensing system to a robotically controlled sheath catheterconfiguration; where FIGS. 17A-F illustrate exemplary sheath instrumentintegrations, and FIGS. 17G-1 and 17G-2 each depict an integration tobuild the exemplary sheath instrument integrations shown in FIGS.17A-17F.

FIG. 18 illustrates a cross sectional view of a bundle of optical fiberwithin the working lumen of a catheter.

FIG. 19 illustrates a robotic surgical system in accordance with someembodiments.

FIG. 20 illustrates an isometric view of an instrument having a guidecatheter in accordance with some embodiments.

FIG. 21 illustrates an isometric view of the instrument of FIG. 20,showing the instrument coupled to a sheath instrument in accordance withsome embodiments.

FIG. 22 illustrates an isometric view of a set of instruments for usewith an instrument driver in accordance with some embodiments.

FIG. 23 illustrates an isometric view of an instrument driver coupledwith a steerable guide instrument and a steerable sheath instrument inaccordance with some embodiments.

FIG. 24 illustrates components of the instrument driver of FIG. 23 inaccordance with some embodiments.

FIG. 25 illustrates the instrument driver of FIG. 24, showing theinstrument driver having a roll motor.

FIG. 26 illustrates components of an instrument driver in accordancewith some embodiments, showing the instrument driver having four motors.

FIG. 27 illustrates an operator control station in accordance with someembodiments.

FIG. 28A illustrates a master input device in accordance with someembodiments.

FIG. 28B illustrates a master input device in accordance with otherembodiments.

FIGS. 29-32 illustrate the manipulation of control or positioningelements adjust the kinematics of a catheter in accordance with variousembodiments, with FIG. 29A illustrating a catheter with tension placedupon a bottom control element, and FIG. 29B illustrating an end view ofthe catheter of FIG. 29A; FIG. 30A illustrating a catheter with tensionplaced upon a left control element, and FIG. 30B illustrating an endview of the catheter of FIG. 30A; FIG. 31A illustrating a catheter withtension placed upon a right control element, and FIG. 31B illustratingan end view of the catheter of FIG. 31A; and FIG. 32A illustrating acatheter with tension placed upon a top control element, and FIG. 32Billustrating an end view of the catheter of FIG. 32A.

FIGS. 33A-33E illustrates different bending configurations of a catheterin accordance with various embodiments.

FIG. 34 illustrates a control system in accordance with someembodiments.

FIG. 35A illustrates a localization sensing system having anelectromagnetic field receiver in accordance with some embodiments.

FIG. 35B illustrates a localization sensing system in accordance withother embodiments.

FIG. 36 illustrates a user interface for a master input device inaccordance with some embodiments.

FIGS. 37-47 illustrate software control schema in accordance withvarious embodiments.

FIG. 48 illustrates forward kinematics and inverse kinematics inaccordance with some embodiments.

FIG. 49 illustrates task coordinates, joint coordinates, and actuationcoordinates in accordance with some embodiments.

FIG. 50 illustrates variables associated with a geometry of a catheterin accordance with some embodiments.

FIG. 51 illustrates a block diagram of a system having a haptic masterinput device.

FIG. 52 illustrates a method for generating a haptic signal inaccordance with some embodiments.

FIG. 53 illustrates a method for converting an operator hand motion to acatheter motion in accordance with some embodiments.

FIG. 54 shows a diagram of where shape information can be integratedinto one example of a robotic control topology.

FIG. 55A shows an example of a control topology augmented by shapeinformation at several possible locations.

FIG. 55B shows a control topology with shape sensing using an observer.

FIG. 55C provides an example of information needed to estimate otherelements of the set.

FIG. 56A represents a shapeable instrument when navigated through anenvironment.

FIG. 56B shows an example of feeding an estimated tip position andorientation and comparing against an input reference position.

FIG. 57A illustrates an example of a modification to apply shapefeedback information into an existing closed loop system to alter a feedforward signal.

FIG. 57B illustrates an alternative closed loop control configurationfor a pure feedback control form that uses an error between the measuredor real shape data and the desired shape.

FIGS. 57C to 57E illustrate examples of a feed back controller combinedwith a feed forward controller to apply shape data.

FIGS. 58A and 58B show examples of using shape data for tracking of ashapeable instrument in the anatomy.

FIGS. 59A and 59B represent examples control relationship where shapesensing occurs after the real instrument is positioned to adjustcatheter mechanics or adapt a kinematic model of the instrument.

FIG. 60 shows an example of overlaying shape data to assess a localenvironment.

FIG. 61A illustrates a general system block diagram for adaption tominimize the differences between predicted and measured positions.

FIG. 61B illustrates an instrument that is prevented from bending as faras expected due to contact with an external object.

FIGS. 62A to 62C illustrate examples of use of a fiber or other elementto measure compression along an axis of a shapeable instrument.

FIGS. 63A to 63C illustrate examples of initial slack in positioningelements of a shapeable instrument.

FIGS. 64A and 64B illustrate examples of using force data on a shapeableinstrument for path planning.

FIGS. 65A and 65B illustrate a normal shape of an instrument andpossible failure modes.

FIG. 65C shows an example of a block diagram to assess fracture.

FIG. 65D shows an example of a block diagram to assess fracture orfailure of a shapeable instrument.

FIGS. 66A and 66B show examples of visual indicators of differentfailure modes.

DETAILED DESCRIPTION

Referring to FIG. 2A, a conventional manually-steerable catheter (1) isdepicted. Pullwires (2) may be selectively tensioned throughmanipulation of a handle (3) on the proximal portion of the catheterstructure to make a more flexible distal portion (5) of the catheterbend or steer controllably. The handle (3) may be coupled, rotatably orslidably, for example, to a proximal catheter structure (34) which maybe configured to be held in the hand, and may be coupled to the elongateportion (35) of the catheter (1). A more proximal, and conventionallyless steerable, portion (4) of the catheter may be configured to becompliant to loads from surrounding tissues (for example, to facilitatepassing the catheter, including portions of the proximal portion,through tortuous pathways such as those formed by the blood vessels),yet less steerable as compared with the distal portion (5).

Referring to FIG. 2B, a robotically-driven steerable catheter (6), hassome similarities with the manually-steerable catheter (1) of FIG. 1 inthat it has pullwires or similar control elements (10) associateddistally with a more flexible section (8) configured to steer or bendwhen the control elements (10) are tensioned in various configurations,as compared with a less steerable proximal portion (7) configured to bestiffer and more resistant to bending or steering. The control elementscan be Flexible tendons, or other mechanical structures that allow forsteering or deflection of the catheter (6). The depicted embodiment ofthe robotically-driven steerable catheter (6) comprises proximal axlesor spindles (9) configured to primarily interface not with fingers orthe hand, but with an electromechanical instrument driver configured tocoordinate and drive, with the help of a computer, each of the spindles(9) to produce precise steering or bending movement of the catheter (6).The spindles (9) may be rotatably coupled to a proximal catheterstructure (32) which may be configured to mount to an electromechanicalinstrument driver apparatus, such as that described in theaforementioned U.S. patent application Ser. No. 11/176,598, and may becoupled to the elongate portion (33) of the catheter (6).

Each of the embodiments depicted in FIGS. 2A and 2B may have a workinglumen (not shown) located, for example, down the central axis of thecatheter body, or may be without such a working lumen. If a workinglumen is formed by the catheter structure, it may extend directly outthe distal end of the catheter, or may be capped or blocked by thedistal tip of the catheter. It is highly useful in many procedures tohave precise information regarding the position of the distal tip ofsuch catheters or other elongate instruments, such as those availablefrom suppliers such as the Ethicon Endosurgery division of Johnson &Johnson, or Intuitive Surgical Corporation. The examples andillustrations that follow are made in reference to arobotically-steerable catheter such as that depicted in FIG. 2B, but aswould be apparent to one skilled in the art, the same principles may beapplied to other elongate instruments, such as the manually-steerablecatheter depicted in FIG. 1, or other elongate instruments, highlyflexible or not, from suppliers such as the Ethicon Endosurgery divisionof Johnson & Johnson, Inc., or Intuitive Surgical, Inc.

Referring to FIGS. 3A-3C, a robotically-steerable catheter (6) isdepicted having an optical fiber (12) positioned along one aspect of thewall of the catheter (6). The fiber is not positioned coaxially with theneutral axis of bending (11) in the bending scenarios depicted in FIGS.3B and 3C. Indeed, with the fiber (12) attached to, or longitudinallyconstrained by, at least two different points along the length of thecatheter (6) body (33) and unloaded from a tensile perspective relativeto the catheter body in a neutral position of the catheter body (33)such as that depicted in FIG. 3A, the longitudinally constrained portionof the fiber (12) would be placed in tension in the scenario depicted inFIG. 3B, while the longitudinally constrained portion of the fiber (12)would be placed in compression in the scenario depicted in FIG. 3C. Suchrelationships are elementary to solid mechanics, but may be applied asdescribed herein with the use of an optical fiber grating to assist inthe determination of deflection of an elongate instrument. As notedabove, the optical fiber grating can comprise a Bragg grating Referringto FIGS. 4A-5D, several different embodiments are depicted. Referring toFIG. 4A, a robotic catheter (6) is depicted having a fiber (12) deployedthrough a lumen (31) which extends from the distal tip of the distalportion (8) of the catheter body (33) to the proximal end of theproximal catheter structure (32). In one embodiment a broadbandreference reflector (not shown) is positioned near the proximal end ofthe fiber in an operable relationship with the optical grating whereinan optical path length is established for each reflector/gratingrelationship comprising the subject fiber grating sensor configuration;additionally, such configuration also comprises a reflectometer (notshown), such as a frequency domain reflectometer, to conduct spectralanalysis of detected reflected portions of light waves.

Constraints (30) may be provided to prohibit axial or longitudinalmotion of the fiber (12) at the location of each constraint (30).Alternatively, the constraints (30) may only constrain the position ofthe fiber (12) relative to the lumen (31) in the location of theconstraints (30). For example, in one variation of the embodimentdepicted in FIG. 4A, the most distal constraint (30) may be configuredto disallow longitudinal or axial movement of the fiber (12) relative tothe catheter body (33) at the location of such constraint (30), whilethe more proximal constraint (30) may merely act as a guide to lift thefiber (12) away from the walls of the lumen (31) at the location of suchproximal constraint (30). In another variation of the embodimentdepicted in FIG. 4A, both the more proximal and more distal constraints(30) may be configured to disallow longitudinal or axial movement of thefiber (12) at the locations of such constraints, and so on. As shown inthe embodiment depicted in FIG. 4A, the lumen (31) in the region of theproximal catheter structure (32) is without constraints to allow forfree longitudinal or axial motion of the fiber relative to the proximalcatheter structure (32). Constraints configured to prohibit relativemotion between the constraint and fiber at a given location may comprisesmall adhesive or polymeric welds, interference fits formed with smallgeometric members comprising materials such as polymers or metals,locations wherein braiding structures are configured with extratightness to prohibit motion of the fiber, or the like. Constraintsconfigured to guide the fiber (12) but to also allow relativelongitudinal or axial motion of the fiber (12) relative to suchconstraint may comprise small blocks, spheres, hemispheres, etc definingsmall holes, generally through the geometric middle of such structures,for passage of the subject fiber (12).

The embodiment of FIG. 4B is similar to that of FIG. 4A, with theexception that there are two additional constraints (30) provided toguide and/or prohibit longitudinal or axial movement of the fiber (12)relative to such constraints at these locations. In one variation, eachof the constraints is a total relative motion constraint, to isolate thelongitudinal strain within each of three “cells” provided by isolatingthe length of the fiber (12) along the catheter body (33) into threesegments utilizing the constraints (30). In another variation of theembodiment depicted in FIG. 4B, the proximal and distal constraints (30)may be total relative motion constraints, while the two intermediaryconstraints (30) may be guide constraints configured to allowlongitudinal or axial relative motion between the fiber (12) and suchconstraints at these intermediary locations, but to keep the fiberaligned near the center of the lumen (31) at these locations.

Referring to FIG. 4C, an embodiment similar to those of FIGS. 4A and 4Bis depicted, with the exception that entire length of the fiber thatruns through the catheter body (33) is constrained by virtue of beingsubstantially encapsulated by the materials which comprise the catheterbody (33). In other words, while the embodiment of FIG. 4C does have alumen (31) to allow free motion of the fiber (12) longitudinally oraxially relative to the proximal catheter structure (32), there is nosuch lumen defined to allow such motion along the catheter body (33),with the exception of the space naturally occupied by the fiber as itextends longitudinally through the catheter body (33) materials whichencapsulate it.

FIG. 4D depicts a configuration similar to that of FIG. 4C with theexception that the lumen (31) extends not only through the proximalcatheter structure (32), but also through the proximal portion (7) ofthe catheter body (33); the distal portion of the fiber (12) which runsthrough the distal portion of the catheter body (33) is substantiallyencapsulated and constrained by the materials which comprise thecatheter body (33).

FIGS. 5A-5D depict embodiments analogous to those depicted in FIGS.4A-D, with the exception that the fiber (12) is positioned substantiallyalong the neutral axis of bending (11) of the catheter body (33), and inthe embodiment of FIG. 5B, there are seven constraints (30) as opposedto the three of the embodiment in FIG. 4B.

Referring to FIG. 6, a cross section of a portion of the catheter body(33) of the configuration depicted in FIG. 4C is depicted, to clearlyillustrate that the fiber (12) is not placed concentrically with theneutral axis (11) of bending for the sample cross section. FIG. 7depicts a similar embodiment, wherein a multi-fiber bundle (13). such asthose available from Luna Technologies, Inc., is positioned within thewall of the catheter rather than a single fiber as depicted in FIG. 6,the fiber bundle (13) comprising multiple, in this embodiment three,individual (e.g., smaller) fibers or fiber cores (14). When a structuresuch as that depicted in FIG. 7 is placed in bending in a configurationsuch as that depicted in FIG. 3B or 3C, the most radially outward (fromthe neutral axis of bending (11)) of the individual fibers (14)experiences more compression or tension than the more radially inwardfibers. Alternatively, in an embodiment such as that depicted in FIG. 8,which shows a cross section of the catheter body (33) portion aconfiguration such as that depicted in FIG. 5C, a multi-fiber bundle(13) is positioned coaxially with the neutral axis of bending (11) forthe catheter (6), and each of three individual fibers (14) within thebundle (13) will experience different degrees of tension and/orcompression in accordance with the bending or steering configuration ofthe subject catheter, as would be apparent to one skilled in the art.For example, referring to FIGS. 9A and 9B (a cross section), at aneutral position, all three individual fibers (14) comprising thedepicted bundle (13) may be in an unloaded configuration. With downwardbending, as depicted in FIGS. 10A and 10B (a cross section), thelowermost two fibers comprising the bundle (13) may be configured toexperience compression, while the uppermost fiber experiences tension.The opposite would happen with an upward bending scenario such as thatdepicted in FIGS. 11A and 11B (cross section).

Indeed, various configurations may be employed, depending upon theparticular application, such as those depicted in FIGS. 12A-12H. Forsimplicity, each of the cross sectional embodiments of FIGS. 12A-12H isdepicted without reference to lumens adjacent the fibers, or constraints(i.e., each of the embodiments of FIGS. 12A-12H are depicted inreference to catheter body configurations analogous to those depicted,for example, in FIGS. 4C and 5C, wherein the fibers are substantiallyencapsulated by the materials comprising the catheter body (33);additional variations comprising combinations and permutations ofconstraints and constraining structures, such as those depicted in FIGS.4A-5D, are within the scope of this invention. FIG. 12A depicts anembodiment having one fiber (12). FIG. 12B depicts a variation havingtwo fibers (12) in a configuration capable of detecting tensionssufficient to calculate three-dimensional spatial deflection of thecatheter portion. FIG. 12C depicts a two-fiber variation with what maybe considered redundancy for detecting bending about a bending axis suchas that depicted in FIG. 12C. FIGS. 12D and 12E depict three-fiberconfigurations configured for detecting three-dimensional spatialdeflection of the subject catheter portion. FIG. 12F depicts a variationhaving four fibers configured to accurately detect three-dimensionalspatial deflection of the subject catheter portion. FIGS. 12G and 12Hdepict embodiments similar to 12B and 12E, respectively, with theexception that multiple bundles of fibers are integrated, as opposed tohaving a single fiber in each location. Each of the embodiments depictedin FIGS. 12A-12H, each of which depicts a cross section of an elongateinstrument comprising at least one optical fiber, may be utilized tofacilitate the determination of bending deflection, torsion, compressionor tension, and/or temperature of an elongate instrument. Suchrelationships may be clarified in reference to FIGS. 13, 14A, and 14B.

In essence, the 3-dimensional position of an elongate member may bedetermined by determining the incremental curvature experienced alongvarious longitudinal sections of such elongate member. In other words,if you know how much an elongate member has curved in space at severalpoints longitudinally down the length of the elongate member, you candetermine the position of the distal portion and more proximal portionsin three-dimensional space by virtue of the knowing that the sectionsare connected, and where they are longitudinally relative to each other.Towards this end, variations of embodiments such as those depicted inFIGS. 12A-12H may be utilized to determine the position of a catheter orother elongate instrument in 3-dimensional space. To determine localcurvatures at various longitudinal locations along an elongateinstrument, fiber optic grating analysis may be utilized.

Referring to FIG. 13, a single optical fiber (12) is depicted havingfour sets of diffraction gratings, each of which may be utilized as alocal deflection sensor. Such a fiber (12) may be interfaced withportions of an elongate instrument, as depicted, for example, in FIGS.12A-12H. A single detector (15) may be utilized to detect and analyzesignals from more than one fiber. With a multi-fiber configuration, suchas those depicted in FIGS. 12B-12H, a proximal manifold structure may beutilized to interface the various fibers with one or more detectors.Interfacing techniques for transmitting signals between detectors andfibers are well known in the art of optical data transmission. Thedetector is operatively coupled with a controller configured todetermine a geometric configuration of the optical fiber and, therefore,at least a portion of the associated elongate instrument (e.g.,catheter) body based on a spectral analysis of the detected reflectedlight signals. Further details are provided in Published US PatentApplication 2006/0013523, the contents of which are fully incorporatedherein by reference.

In the single fiber embodiment depicted in FIG. 13, each of thediffraction gratings has a different spacing (d1, d2, d3, d4), and thusa proximal light source for the depicted single fiber and detector maydetect variations in wavelength for each of the “sensor” lengths (L10,L20, L30, L40). Thus, given determined length changes at each of the“sensor” lengths (L10, L20, L30, L40), the longitudinal positions of the“sensor” lengths (L10, L20, L30, L40), and a known configuration such asthose depicted in cross section in FIGS. 12A-12H, the deflection and/orposition of the associated elongate instrument in space may bedetermined. One of the challenges with a configuration such as thatdepicted in FIG. 13 is that a fairly broad band emitter and broad bandtunable detector must be utilized proximally to capture lengthdifferentiation data from each of the sensor lengths, potentiallycompromising the number of sensor lengths that may be monitored, etc.Regardless, several fiber (12) and detector (15) configurations such asthat depicted in FIG. 13 may comprise embodiments such as those depictedin FIGS. 12A-12H to facilitate determination of three-dimensionalpositioning of an elongate medical instrument.

In another embodiment of a single sensing fiber, depicted in FIG. 14A,various sensor lengths (L50, L60, L70, L80) may be configured to eachhave the same grating spacing, and a more narrow band source may beutilized with some sophisticated analysis. as described, for example, in“Sensing Shape—Fiber-Bragg-grating sensor arrays monitor shape at highresolution,” SPIE's OE Magazine, September, 2005, pages 18-21,incorporated by reference herein in its entirety, to monitor elongationat each of the sensor lengths given the fact that such sensor lengthsare positioned at different positions longitudinally (L1, L2, L3, L4)away from the proximal detector (15). In another (related) embodiment,depicted in FIG. 14B, a portion of a given fiber, such as the distalportion, may have constant gratings created to facilitatehigh-resolution detection of distal lengthening or shortening of thefiber. Such a constant grating configuration would also be possible withthe configurations described in the aforementioned scientific journalarticle.

Referring to FIGS. 15A and 15B, temperature may be sensed utilizingFiber-Bragg grating sensing in embodiments similar to those depicted inFIGS. 13 and 14A-B. Referring to FIG. 15A, a single fiber protrudesbeyond the distal tip of the depicted catheter (6) and is unconstrained,or at least less constrained, relative to other surrounding structuresso that the portion of the depicted fiber is free to change in lengthwith changes in temperature. With knowledge of the thermal expansion andcontraction qualities of the small protruding fiber portion, and one ormore Bragg diffraction gratings in such protruding portion, the changesin length may be used to extrapolate changes in temperature and thus beutilized for temperature sensing. Referring to FIG. 15B, a small cavity(21) or lumen may be formed in the distal portion of the catheter body(33) to facilitate free movement of the distal portion (22) of the fiber(12) within such cavity (21) to facilitate temperature sensing distallywithout the protruding fiber depicted in FIG. 15A.

As will be apparent to those skilled in the art, the fibers in theembodiments depicted herein will provide accurate measurements oflocalized length changes in portions of the associated catheter orelongate instrument only if such fiber portions are indeed coupled insome manner to the nearby portions of the catheter or elongateinstrument. In one embodiment, it is desirable to have the fiber orfibers intimately coupled with or constrained by the surroundinginstrument body along the entire length of the instrument, with theexception that one or more fibers may also be utilized to sensetemperature distally, and may have an unconstrained portion, as in thetwo scenarios described in reference to FIGS. 15A and 15B. In oneembodiment, for example, each of several deflection-sensing fibers mayterminate in a temperature sensing portion, to facilitate positiondetermination and highly localized temperature sensing and comparison atdifferent aspects of the distal tip of an elongate instrument. Inanother embodiment, the proximal portions of the fiber(s) in the lessbendable catheter sections are freely floating within the catheter body,and the more distal/bendable fiber portions intimately coupled, tofacilitate high-precision monitoring of the bending within the distal,more flexible portion of the catheter or elongate instrument.

Referring to FIGS. 16A, 16B, and 16D, a catheter-like robotic guideinstrument integration embodiment is depicted with three optical fibers(12) and a detector (15) for detecting catheter bending and distal tipposition. FIG. 16C depicts and embodiment having four optical fibers(12) for detecting catheter position. FIG. 16D depicts an integration tobuild such embodiments. As shown in FIG. 16D, in step “E+”, mandrels foroptical fibers are woven into a braid layer, subsequent to which (step“F”) Bragg-grated optical fibers are positioned in the cross sectionalspace previously occupied by such mandrels (after such mandrels areremoved). The geometry of the mandrels relative to the fibers selectedto occupy the positions previously occupied by the mandrels after themandrels are removed preferably is selected based upon the level ofconstraint desired between the fibers (12) and surrounding catheter body(33) materials. For example, if a highly-constrained relationship,comprising substantial encapsulation, is desired, the mandrels willclosely approximate the size of the fibers. If a moreloosely-constrained geometric relationship is desired, the mandrels maybe sized up to allow for relative motion between the fibers (12) and thecatheter body (33) at selected locations, or a tubular member, such as apolyimide or PTFE sleeve, may be inserted subsequent to removal of themandrel, to provide a “tunnel” with clearance for relative motion of thefiber, and/or simply a layer of protection between the fiber and thematerials surrounding it which comprise the catheter or instrument body(33). Similar principles may be applied in embodiments such as thosedescribed in reference to FIGS. 17A-17G.

Referring to FIGS. 17A-F, two sheath instrument integrations aredepicted, each comprising a single optical fiber (12). FIG. 17G depictsan integration to build such embodiments. As shown in FIG. 16D, in step“B”, a mandrel for the optical fiber is placed, subsequent to which(step “K”) a Bragg-grated optical fiber is positioned in the crosssectional space previously occupied by the mandrel (after such mandrelis removed).

Referring to FIG. 18, in another embodiment, a bundle (13) of fibers(14) may be placed down the working lumen of an off-the-shelf roboticcatheter (guide or sheath instrument type) such as that depicted in FIG.18, and coupled to the catheter in one or more locations, with aselected level of geometric constraint, as described above, to provide3-D spatial detection.

Tension and compression loads on an elongate instrument may be detectedwith common mode deflection in radially-outwardly positioned fibers, orwith a single fiber along the neutral bending axis. Torque may bedetected by sensing common mode additional tension (in addition, forexample, to tension and/or compression sensed by, for example, a singlefiber coaxial with the neutral bending axis) in outwardly-positionedfibers in configurations such as those depicted in FIGS. 12A-H.

In another embodiment, the tension elements utilized to actuate bending,steering, and/or compression of an elongate instrument, such as asteerable catheter, may comprise optical fibers with gratings, ascompared with more conventional metal wires or other structures, andthese fiber optic tension elements may be monitored for deflection asthey are loaded to induce bending/steering to the instrument. Suchmonitoring may be used to prevent overstraining of the tension elements,and may also be utilized to detect the position of the instrument as awhole, as per the description above.

Referring to FIG. 19, one embodiment of a robotic catheter system 32,includes an operator control station 2 located remotely from anoperating table 22, to which a instrument driver 16 and instrument 18are coupled by a instrument driver mounting brace 20. A communicationlink 14 transfers signals between the operator control station 2 andinstrument driver 16. The instrument driver mounting brace 20 of thedepicted embodiment is a relatively simple, arcuate-shaped structuralmember configured to position the instrument driver 16 above a patient(not shown) lying on the table 22.

FIGS. 20 and 21 depict isometric views of respective embodiments ofinstruments configured for use with an embodiment of the instrumentdriver (16), such as that depicted in FIG. 19. FIG. 20 depicts aninstrument (18) embodiment without an associated coaxial sheath coupledat its midsection. FIG. 21 depicts a set of two instruments (28),combining an embodiment like that of FIG. 20 with a coaxially coupledand independently controllable sheath instrument (30). To distinguishthe non-sheath instrument (18) from the sheath instrument (30) in thecontext of this disclosure, the “non-sheath” instrument may also betermed the “guide” instrument (18).

Referring to FIG. 22, a set of instruments (28), such as those in FIG.21, is depicted adjacent an instrument driver (16) to illustrate anexemplary mounting scheme. The sheath instrument (30) may be coupled tothe depicted instrument driver (16) at a sheath instrument interfacesurface (38) having two mounting pins (42) and one interface socket (44)by sliding the sheath instrument base (46) over the pins (42).Similarly, and preferably simultaneously, the guide instrument (18) base(48) may be positioned upon the guide instrument interface surface (40)by aligning the two mounting pins (42) with alignment holes in the guideinstrument base (48). As will be appreciated, further steps may berequired to lock the instruments (18, 30) into place upon the instrumentdriver (16).

In FIG. 23, an instrument driver (16) is depicted as interfaced with asteerable guide instrument (18) and a steerable sheath instrument (30).FIG. 24 depicts an embodiment of the instrument driver (16), in whichthe sheath instrument interface surface (38) remains stationary, andrequires only a simple motor actuation in order for a sheath to besteered using an interfaced control element via a control elementinterface assembly (132). This may be accomplished with a simple cableloop about a sheath socket drive pulley (272) and a capstan pulley (notshown), which is fastened to a motor, similar to the two upper motors(242) (visible in FIG. 24). The drive motor for the sheath socket driveschema is hidden under the linear bearing interface assembly.

The drive schema for the four guide instrument interface sockets (270)is more complicated, due in part to the fact that they are coupled to acarriage (240) configured to move linearly along a linear bearinginterface (250) to provide for motor-driven insertion of a guideinstrument toward the patient relative to the instrument driver,hospital table, and sheath instrument. Various conventional cabletermination and routing techniques are utilized to accomplish apreferably high-density instrument driver structure with the carriage(240) mounted forward of the motors for a lower profile patient-sideinterface.

Still referring to FIG. 24, the instrument driver (16) is rotatablymounted to an instrument driver base (274), which is configured tointerface with an instrument driver mounting brace (not shown), such asthat depicted in FIG. 19, or a movable setup joint construct (notshown). Rotation between the instrument driver base (274) and aninstrument driver base plate (276) to which it is coupled is facilitatedby a heavy-duty flanged bearing structure (278). The flanged bearingstructure (278) is configured to allow rotation of the body of theinstrument driver (16) about an axis approximately coincident with thelongitudinal axis of a guide instrument (not shown) when the guideinstrument is mounted upon the instrument driver (16) in a neutralposition. This rotation preferably is automated or powered by a rollmotor (280) and a simple roll cable loop (286), which extends aroundportions of the instrument driver base plate and terminates as depicted(282,284). Alternatively, roll rotation may be manually actuated andlocked into place with a conventional clamping mechanism. The roll motor(280) position is more easily visible in FIG. 25.

FIG. 26 illustrates another embodiment of an instrument driver,including a group of four motors (290). Each motor (290) has anassociated high-precision encoder for controls purposes and beingconfigured to drive one of the four guide instrument interface sockets(270), at one end of the instrument driver. Another group of two motors(one hidden, one visible—288) with encoders (292) are configured todrive insertion of the carriage (240) and the sheath instrumentinterface socket (268).

Referring to FIG. 27, an operator control station is depicted showing acontrol button console (8), a computer (6), a computer control interface(10), such as a mouse, a visual display system (4) and a master inputdevice (12). In addition to “buttons” on the button console (8)footswitches and other known user control interfaces may be utilized toprovide an operator interface with the system controls.

Referring to FIG. 28A, in one embodiment, the master input device (12)is a multidegree-of-freedom device having multiple joints and associatedencoders (306). An operator interface (217) is configured forcomfortable interfacing with the human fingers. The depicted embodimentof the operator interface (217) is substantially spherical. Further, themaster input device may have integrated haptics capability for providingtactile feedback to the user.

Another embodiment of a master input device (12) is depicted in FIG. 28Bhaving a similarly-shaped operator interface (217). Suitable masterinput devices are available from manufacturers such as Sensible DevicesCorporation under the trade name “Phanto™”, or Force Dimension under thetrade name “Omega™”. In one embodiment featuring an Omega-type masterinput device, the motors of the master input device are utilized forgravity compensation. In other words, when the operator lets go of themaster input device with his hands, the master input device isconfigured to stay in position, or hover around the point at which iswas left, or another predetermined point, without gravity taking thehandle of the master input device to the portion of the master inputdevice's range of motion closest to the center of the earth. In anotherembodiment, haptic feedback is utilized to provide feedback to theoperator that he has reached the limits of the pertinent instrumentworkspace. In another embodiment, haptic feedback is utilized to providefeedback to the operator that he has reached the limits of the subjecttissue workspace when such workspace has been registered to theworkspace of the instrument (i.e., should the operator be navigating atool such as an ablation tip with a guide instrument through a 3-D modelof a heart imported, for example, from CT data of an actual heart, themaster input device is configured to provide haptic feedback to theoperator that he has reached a wall or other structure of the heart asper the data of the 3-D model, and therefore help prevent the operatorfrom driving the tool through such wall or structure without at leastfeeling the wall or structure through the master input device). Inanother embodiment, contact sensing technologies configured to detectcontact between an instrument and tissue may be utilized in conjunctionwith the haptic capability of the master input device to signal theoperator that the instrument is indeed in contact with tissue.

Referring to FIGS. 29-32, the basic kinematics of a catheter with fourcontrol elements is reviewed.

Referring to FIGS. 29A-B, as tension is placed only upon the bottomcontrol element (312), the catheter bends downward, as shown in FIG.29A. Similarly, pulling the left control element (314) in FIGS. 30A-Bbends the catheter left, pulling the right control element (310) inFIGS. 31A-B bends the catheter right, and pulling the top controlelement (308) in FIGS. 32A-B bends the catheter up. As will be apparentto those skilled in the art, well-known combinations of applied tensionabout the various control elements results in a variety of bendingconfigurations at the tip of the catheter member (90). One of thechallenges in accurately controlling a catheter or similar elongatemember with tension control elements is the retention of tension incontrol elements, which may not be the subject of the majority of thetension loading applied in a particular desired bending configuration.If a system or instrument is controlled with various levels of tension,then losing tension, or having a control element in a slackconfiguration, can result in an unfavorable control scenario.

Referring to FIGS. 33A-E, a simple scenario is useful in demonstratingthis notion. As shown in FIG. 33A, a simple catheter (316) steered withtwo control elements (314. 310) is depicted in a neutral position. Ifthe left control element (314) is placed into tension greater than thetension, if any, which the right control element (310) experiences, thecatheter (316) bends to the left, as shown in FIG. 33B. If a change ofdirection is desired, this paradigm needs to reverse, and the tension inthe right control element (310) needs to overcome that in the leftcontrol element (314). At the point of a reversal of direction likethis, where the tension balance changes from left to right, withoutslack or tension control, the right most control element (314) maygather slack which needs to be taken up before precise control can bereestablished. Subsequent to a “reeling in” of slack which may bepresent, the catheter (316) may be may be pulled in the oppositedirection, as depicted in FIGS. 33CE, without another slack issue from acontrols perspective until a subsequent change in direction.

The above-described instrument embodiments present various techniquesfor managing tension control in various guide instrument systems havingbetween two and four control elements. For example, in one set ofembodiments, tension may be controlled with active independenttensioning of each control element in the pertinent guide catheter viaindependent control element interface assemblies (132) associated withindependently-controlled guide instrument interface sockets (270) on theinstrument driver (16). Thus, tension may be managed by independentlyactuating each of the control element interface assemblies (132) in afour-control-element embodiment, a three-control-element embodiment, ora two-control-element embodiment.

In another set of embodiments, tension may be controlled with activeindependent tensioning with a split carriage design. For example, asplit carriage with two independent linearly movable portions, may beutilized to actively and independently tension each of the two controlelement interface assemblies (132), each of which is associated with twodimensions of a given degree of freedom. For example, one interfaceassembly can include + and − pitch, with + and − yaw on the otherinterface assembly, where slack or tension control provided for pitch byone of the linearly movable portions (302) of the split carriage (296),and slack or tension control provided for yaw by the other linearlymovable portion (302) of the split carriage (296).

Similarly, slack or tension control for a single degree of freedom, suchas yaw or pitch, may be provided by a single-sided split carriagedesign, with the exception that only one linearly movable portion wouldbe required to actively tension the single control element interfaceassembly of an instrument.

In another set of embodiments, tensioning may be controlled withspring-loaded idlers configured to keep the associated control elementsout of slack. The control elements preferably are pre-tensioned in eachembodiment to prevent slack and provide predictable performance. Indeed,in yet another set of embodiments, pre-tensioning may form the mainsource of tension management. In the case of embodiments only havingpre-tensioning or spring-loaded idler tensioning, the control system mayneed to be configured to reel in bits of slack at certain transitionpoints in catheter bending, such as described above in relation to FIGS.33A and 33B.

To accurately coordinate and control actuations of various motors withinan instrument driver from a remote operator control station such as thatdepicted in FIG. 19, an advanced computerized control and visualizationsystem is preferred. While the control system embodiments that followare described in reference to a particular control systems interface,namely the SimuLink™ and XPC™ control interfaces available from TheMathworks Inc., and PC-based computerized hardware configurations, manyother configurations may be utilized, including various pieces ofspecialized hardware, in place of more flexible software controlsrunning on PC-based systems.

Referring to FIG. 34, an overview of an embodiment of a controls systemflow is depicted. A master computer (400) running master input devicesoftware, visualization software, instrument localization software, andsoftware to interface with operator control station buttons and/orswitches is depicted: In one embodiment, the master input devicesoftware is a proprietary module packaged with an off-the-shelf masterinput device system, such as the Phantom™ from Sensible DevicesCorporation, which is configured to communicate with the Phantom™hardware at a relatively high frequency as prescribed by themanufacturer. Other suitable master input devices, such as that (12)depicted in FIG. 28B are available from suppliers such as ForceDimension of Lausanne, Switzerland. The master input device (12) mayalso have haptics capability to facilitate feedback to the operator, andthe software modules pertinent to such functionality may also beoperated on the master computer (400). Preferred embodiments of hapticsfeedback to the operator are discussed in further detail below.

The term “localization” is used in the art in reference to systems fordetermining and/or monitoring the position of objects, such as medicalinstruments, in a reference coordinate system. In one embodiment, theinstrument localization software is a proprietary module packaged withan off-the-shelf or custom instrument position tracking system, such asthose available from Ascension Technology Corporation, Biosense Webster,Inc., Endocardial Solutions, Inc., Boston Scientific (EP Technologies),Medtronic, Inc., and others. Such systems may be capable of providingnot only real-time or near real-time positional information, such asX-Y-Z coordinates in a Cartesian coordinate system, but also orientationinformation relative to a given coordinate axis or system. For example,such systems can employ an electromagnetic based system (e.g., usingelectromagnetic coils inside a device or catheter body). Informationregarding one electromagnetic based system can be found on:http://www.biosensewebster.com/products/navigation/carto3.aspx. Therelevant portions of which are incorporated by reference.

Some of the commercially-available localization systems useelectromagnetic relationships to determine position and/or orientation,while others, such as some of those available from EndocardialSolutions. Inc.—St Jude Medical, utilize potential difference orvoltage, as measured between a conductive sensor located on thepertinent instrument and conductive portions of sets of patches placedagainst the skin, to determine position and/or orientation. Referring toFIGS. 35A and 35B, various localization sensing systems may be utilizedwith the various embodiments of the robotic catheter system disclosedherein. In other embodiments not comprising a localization system todetermine the position of various components, kinematic and/or geometricrelationships between various components of the system may be utilizedto predict the position of one component relative to the position ofanother. Some embodiments may utilize both localization data andkinematic and/or geometric relationships to determine the positions ofvarious components. Information regarding impedance based systems can befound on:http://www.sjmprofessional.com/Products/US/Mapping-and-Visualization/EnSite-Velocity.aspx.The relevant portions of which are incorporated by reference.

As shown in FIG. 35A, one preferred localization system comprises anelectromagnetic field transmitter (406) and an electromagnetic fieldreceiver (402) positioned within the central lumen of a guide catheter(90). The transmitter (406) and receiver (402) are interfaced with acomputer operating software configured to detect the position of thedetector relative to the coordinate system of the transmitter (406) inreal or near-real time with high degrees of accuracy. Referring to FIG.35B, a similar embodiment is depicted with a receiver (404) embeddedwithin the guide catheter (90) construction. Preferred receiverstructures may comprise three or more sets of very small coils spatiallyconfigured to sense orthogonal aspects of magnetic fields emitted by atransmitter. Such coils may be embedded in a custom configuration withinor around the walls of a preferred catheter construct. For example, inone embodiment, two orthogonal coils are embedded within a thinpolymeric layer at two slightly flattened surfaces of a catheter (90)body approximately ninety degrees orthogonal to each other about thelongitudinal axis of the catheter (90) body, and a third coil isembedded in a slight polymer-encapsulated protrusion from the outside ofthe catheter (90) body, perpendicular to the other two coils. Due to thevery small size of the pertinent coils, the protrusion of the third coilmay be minimized. Electronic leads for such coils may also be embeddedin the catheter wall, down the length of the catheter body to aposition, preferably adjacent an instrument driver, where they may berouted away from the instrument to a computer running localizationsoftware and interfaced with a pertinent transmitter.

In another similar embodiment (not shown), one or more conductive ringsmay be electronically connected to a potential-difference-basedlocalization/orientation system, along with multiple sets, preferablythree sets, of conductive skin patches, to provide localization and/ororientation data utilizing a system such as those available fromEndocardial Solutions—St. Jude Medical. The one or more conductive ringsmay be integrated into the walls of the instrument at variouslongitudinal locations along the instrument, or set of instruments. Forexample, a guide instrument may have several conductive ringslongitudinally displaced from each other toward the distal end of theguide instrument, while a coaxially-coupled sheath instrument maysimilarly have one or more conductive rings longitudinally displacedfrom each other toward the distal end of the sheath instrument—toprovide precise data regarding the location and/or orientation of thedistal ends of each of such instruments.

Referring back to FIG. 34, in one embodiment, visualization softwareruns on the master computer (400) to facilitate real-time driving andnavigation of one or more steerable instruments. In one embodiment,visualization software provides an operator at an operator controlstation, such as that depicted in FIG. 19 (2), with a digitized“dashboard” or “windshield” display to enhance instinctive drivabilityof the pertinent instrumentation within the pertinent tissue structures.Referring to FIG. 36, a simple illustration is useful to explain oneembodiment of a preferred relationship between visualization andnavigation with a master input device (12). In the depicted embodiment,two display views (410, 412) are shown. One preferably represents aprimary (410) navigation view, and one may represent a secondary (412)navigation view. To facilitate instinctive operation of the system, itis preferable to have the master input device coordinate system at leastapproximately synchronized with the coordinate system of at least one ofthe two views. Further, it is preferable to provide the operator withone or more secondary views which may be helpful in navigating throughchallenging tissue structure pathways and geometries.

Using the operation of an automobile as an example, if the master inputdevice is a steering wheel and the operator desires to drive a car in aforward direction using one or more views, his first priority is likelyto have a view straight out the windshield, as opposed to a view out theback window, out one of the side windows, or from a car in front of thecar that he is operating. The operator might prefer to have the forwardwindshield view as his primary display view, such that a right turn onthe steering wheel takes him right as he observes his primary display, aleft turn on the steering wheel takes him left, and so forth. If theoperator of the automobile is trying to park the car adjacent anothercar parked directly in front of him, it might be preferable to also havea view from a camera positioned, for example, upon the sidewalk aimedperpendicularly through the space between the two cars (one driven bythe operator and one parked in front of the driven car), so the operatorcan see the gap closing between his car and the car in front of him ashe parks. While the driver might not prefer to have to completelyoperate his vehicle with the sidewalk: perpendicular camera view as hissole visualization for navigation purposes, this view is helpful as asecondary view.

Referring still to FIG. 36, if an operator is attempting to navigate asteerable catheter in order to, for example, contact a particular tissuelocation with the catheter's distal tip, a useful primary navigationview (410) may comprise a three dimensional digital model of thepertinent tissue structures (414) through which the operator isnavigating the catheter with the master input device (12), along with arepresentation of the catheter distal tip location (416) as viewed alongthe longitudinal axis of the catheter near the distal tip. Thisembodiment illustrates a representation of a targeted tissue structurelocation (418), which may be desired in addition to the tissue digitalmodel (414) information. A useful secondary view (412), displayed upon adifferent monitor, in a different window upon the same monitor, orwithin the same user interface window, for example, comprises anorthogonal view depicting the catheter tip representation (416), andalso perhaps a catheter body representation (420), to facilitate theoperator's driving of the catheter tip toward the desired targetedtissue location (418).

In one embodiment, subsequent to development and display of a digitalmodel of pertinent tissue structures, an operator may select one primaryand at least one secondary view to facilitate navigation of theinstrumentation. By selecting which view is a primary view, the user canautomatically toggle a master input device (12) coordinate system tosynchronize with the selected primary view. In an embodiment with theleftmost depicted view (410) selected as the primary view, to navigatetoward the targeted tissue site (418), the operator should manipulatethe master input device (12) forward, to the right, and down. The rightview will provide valued navigation information, but will not be asinstinctive from a “driving” perspective.

To illustrate: if the operator wishes to insert the catheter tip towardthe targeted tissue site (418) watching only the rightmost view (412)without the master input device (12) coordinate system synchronized withsuch view, the operator would have to remember that pushing straightahead on the master input device will make the distal tip representation(416) move to the right on the rightmost display (412). Should theoperator decide to toggle the system to use the rightmost view (412) asthe primary navigation view, the coordinate system of the master inputdevice (12) is then synchronized with that of the rightmost view (412),enabling the operator to move the catheter tip (416) closer to thedesired targeted tissue location (418) by manipulating the master inputdevice (12) down and to the right.

The synchronization of coordinate systems described herein may beconducted using fairly conventional mathematic relationships. Forexample, in one embodiment, the orientation of the distal tip of thecatheter may be measured using a 6 axis position sensor system such asthose available from Ascension Technology Corporation, Biosense Webster,Inc., Endocardial Solutions, Inc., Boston Scientific (EP Technologies),and others. A 3-axis coordinate frame. C, for locating the distal tip ofthe catheter, is constructed from this orientation information. Theorientation information is used to construct the homogeneoustransformation matrix, T^(G0) _(Gref), which transforms a vector in theCatheter coordinate frame “C” to the fixed Global coordinate frame “G”in which the sensor measurements are done (the Subscript G_(ref) andsuperscript C_(ref) are used to represent the O'th, or initial, step).As a registration step, the computer graphics view of the catheter isrotated until the master input and the computer graphics view of thecatheter distal tip motion are coordinated and aligned with the cameraview of the graphics scene. The 3-axis coordinate frame transformationmatrix T^(G0) _(Gref) for the camera position of this initial view isstored (subscripts G_(ref) and superscript C_(ref) stand for the globaland camera “reference” views). The corresponding catheter “referenceview” matrix for the catheter coordinates is obtained as:T _(Cref) ^(C0) =T _(G0) ^(C0) T _(Gref) ^(G0) T _(Cref) ^(Gref)=(T_(C0) ^(G0))⁻¹ T _(Gref) ^(G0) T _(C1) ^(G1)

Also note that the catheter's coordinate frame is fixed in the globalreference frame G, thus the transformation matrix between the globalframe and the catheter frame is the same in all views, i.e., T_(C0)^(G0)=T_(Cref) ^(Gref)=T_(Ci) ^(Gi) for any arbitrary view i. Thecoordination between primary view and master input device coordinatesystems is achieved by transforming the master input as follows: Givenany arbitrary computer graphics view of the representation, e.g. thei'th view, the 3-axis coordinate frame transformation matrix T_(Gi)^(G0) of the camera view of the computer graphics scene is obtained fromthe computer graphics software. The corresponding cathetertransformation matrix is computed in a similar manner as above:T _(Ci) ^(C0) =T _(G0) ^(C0) T _(Gi) ^(G0) T _(Ci) ^(Gi)=(T _(C0)^(G0))⁻¹ T _(gi) ^(G0) T _(Ci) ^(Gi)

The transformation that needs to be applied to the master input whichachieves the view coordination is the one that transforms from thereference view that was registered above, to the current ith view, i.e.,T_(Cref) ^(Ci). Using the previously computed quantities above, thistransform is computed as:T _(Cref) ^(Ci) =T _(C0) ^(Ci) T _(Cref) ^(C0)

The master input is transformed into the commanded catheter input byapplication of the transformation T_(Cref) ^(Ci). Given a command input

${r_{master} = \begin{bmatrix}x_{master} \\y_{master} \\y_{master}\end{bmatrix}},$one may calculate:

$r_{catheter} = {\begin{bmatrix}x_{catheter} \\y_{catheter} \\y_{catheter}\end{bmatrix} = {{T_{Cref}^{Ci}\begin{bmatrix}x_{master} \\y_{master} \\y_{master}\end{bmatrix}}.}}$

Under such relationships, coordinate systems of the primary view andmaster input device may be aligned for instinctive operation.

Referring back to embodiment of FIG. 34, the master computer (400) alsocomprises software and hardware interfaces to operator control stationbuttons, switches, and other input devices which may be utilized, forexample, to “freeze” the system by functionally disengaging the masterinput device as a controls input, or provide toggling between variousscaling ratios desired by the operator for manipulated inputs at themaster input device (12). The master computer (400) has two separatefunctional connections with the control and instrument driver computer(422): one (426) for passing controls and visualization relatedcommands, such as desired XYZ) in the catheter coordinate system)commands, and one (428) for passing safety signal commands. Similarly,the control and instrument driver computer (422) has two separatefunctional connections with the instrument and instrument driverhardware (424): one (430) for passing control and visualization relatedcommands such as required-torque-related voltages to the amplifiers todrive the motors and encoders, and one (432) for passing safety signalcommands.

In one embodiment, the safety signal commands represent a simple signalrepeated at very short intervals, such as every 10 milliseconds, suchsignal chain being logically read as “system is ok, amplifiers stayactive”. If there is any interruption in the safety signal chain, theamplifiers are logically toggled to inactive status and the instrumentcannot be moved by the control system until the safety signal chain isrestored. Also shown in the signal flow overview of FIG. 34 is a pathway(434) between the physical instrument and instrument driver hardwareback to the master computer to depict a closed loop system embodimentwherein instrument localization technology, such as that described inreference to FIGS. 35A-B, is utilized to determine the actual positionof the instrument to minimize navigation and control error, as describedin further detail below

FIGS. 37-47 depict various aspects of one embodiment of a SimuLink™software control schema for an embodiment of the physical system, withparticular attention to an embodiment of a “master following mode.” Inthis embodiment, an instrument is driven by following instructions froma master input device, and a motor servo loop embodiment, whichcomprises key operational functionality for executing upon commandsdelivered from the master following mode to actuate the instrument.

FIG. 37 depicts a high-level view of an embodiment wherein anyone ofthree modes may be toggled to operate the primary servo loop (436). Inidle mode (438), the default mode when the system is started up, all ofthe motors are commanded via the motor servo loop (436) to servo abouttheir current positions, their positions being monitored with digitalencoders associated with the motors. In other words, idle mode (438)deactivates the motors, while the remaining system stays active. Thus,when the operator leaves idle mode, the system knows the position of therelative components. In auto home mode (440), cable loops within anassociated instrument driver, such as that depicted in FIG. 23, arecentered within their cable loop range to ensure substantiallyequivalent range of motion of an associated instrument in bothdirections for a various degree of freedom, such as + and − directionsof pitch or yaw, when loaded upon the instrument driver. This is a setupmode for preparing an instrument driver before an instrument is engaged.

In master following mode (442), the control system receives signals fromthe master input device, and in a closed loop embodiment from both amaster input device and a localization system, and forwards drivesignals to the primary servo loop (436) to actuate the instrument inaccordance with the forwarded commands. Aspects of this embodiment ofthe master following mode (442) are depicted in further detail in FIGS.42-124. Aspects of the primary servo loop and motor servo block (444)are depicted in further detail in FIGS. 38-41.

Referring to FIG. 42, a more detailed functional diagram of anembodiment of master following mode (442) is depicted. As shown in FIG.42, the inputs to functional block (446) are XYZ position of the masterinput device in the coordinate system of the master input device which,per a setting in the software of the master input device may be alignedto have the same coordinate system as the catheter, and localization XYZposition of the distal tip of the instrument as measured by thelocalization system in the same coordinate system as the master inputdevice and catheter. Referring to FIG. 43 for a more detailed view offunctional block (446) of FIG. 42, a switch (460) is provided at blockto allow switching between master inputs for desired catheter position,to an input interface (462) through which an operator may command thatthe instrument go to a particular XYZ location in space. Variouscontrols features may also utilize this interface to provide an operatorwith, for example, a menu of destinations to which the system shouldautomatically drive an instrument, etc. Also depicted in FIG. 43 is amaster scaling functional block (451) which is utilized to scale theinputs coming from the master input device with a ratio selectable bythe operator. The command switch (460) functionality includes a low passfilter to weight commands switching between the master input device andthe input interface (462), to ensure a smooth transition between thesemodes.

Referring back to FIG. 42, desired position data in XYZ terms is passedto the inverse kinematics block (450) for conversion to pitch, yaw, andextension (or “insertion”) terms in accordance with the predictedmechanics of materials relationships inherent in the mechanical designof the instrument.

The kinematic relationships for many catheter instrument embodiments maybe modeled by applying conventional mechanics relationships. In summary,a control-element-steered catheter instrument is controlled through aset of actuated inputs. In a four-control-element catheter instrument,for example, there are two degrees of motion actuation, pitch and yaw,which both have + and − directions. Other motorized tensionrelationships may drive other instruments, active tensioning, orinsertion or roll of the catheter instrument. The relationship betweenactuated inputs and the catheter's end point position as a function ofthe actuated inputs is referred to as the “kinematics” of the catheter.

Referring to FIG. 48, the “forward kinematics” expresses the catheter'send-point position as a function of the actuated inputs while the“inverse kinematics” expresses the actuated inputs as a function of thedesired end-point position. Accurate mathematical models of the forwardand inverse kinematics are essential for the control of a roboticallycontrolled catheter system. For clarity, the kinematics equations arefurther refined to separate out common elements, as shown in FIG. 48.The basic kinematics describes the relationship between the taskcoordinates and the joint coordinates. In such case, the taskcoordinates refer to the position of the catheter end-point while thejoint coordinates refer to the bending (pitch and yaw, for example) andlength of the active catheter. The actuator kinematics describes therelationship between the actuation coordinates and the jointcoordinates. The task, joint, and bending actuation coordinates for therobotic catheter are illustrated in FIG. 49. By describing thekinematics in this way we can separate out the kinematics associatedwith the catheter structure, namely the basic kinematics, from thoseassociated with the actuation methodology.

An inverse kinematic model translates intended device motion into thecommands that will adjust the actuator and/or control element toposition the shapeable instrument as desired. Referring back to FIG. 1B,the shapeable instrument kinematics are the mathematical relationshipsbetween the task space description of the instrument (e.g., tipposition) and the configuration space description of the instrument(e.g., shape). Specifically, the inverse kinematics (task toconfiguration space) are used as part of the chain that translatesdesired tip positions into actuator commands (leading to displacementsof the control elements) that move tip position of the actual device forreaching a desired tip position.

These inverse kinematic algorithms are derived based upon certainassumptions about how the shapeable instrument moves. Examples of theseassumptions include but are not limited to: 1) Each catheter segmentbends in a constant curvature arc; 2) Each catheter segment bends withina single plane; 3) Some catheter segments have fixed (constant) lengths;4) Some catheter segments have variable (controllable) lengths;

The development of the catheter's kinematics model is derived using afew essential assumptions. Included are assumptions that the catheterstructure is approximated as a simple beam in bending from a mechanicsperspective, and that control elements, such as thin tension wires,remain at a fixed distance from the neutral axis and thus impart auniform moment along the length of the catheter.

In addition to the above assumptions, the geometry and variables shownin FIG. 50 are used in the derivation of the forward and inversekinematics. The basic forward kinematics, relating the catheter taskcoordinates (X_(C), Y_(C), Z_(C)) to the joint coordinates (□_(pitch),(□_(pitch), L), is given as follows:X _(c) =w cos(θ)Y _(c) =R sin(α)Z _(c) =w sin(θ)

Where

w = R(1 − cos (α)) α = [(ϕ_(pitch))² + (ϕ_(yaw))²]^(1/2)(total  bending)$R = {\frac{L}{\alpha}\left( {{bend}\mspace{14mu}{radius}} \right)}$θ = a tan  2(ϕ_(pitch), ϕ_(yaw))(roll  angle)

The actuator forward kinematics, relating the joint coordinates(□_(pitch), □_(pitch), L) to the actuator coordinates (□L_(x), □L_(z),L)is given as follows:

$\phi_{yaw} = \frac{2\;\Delta\; L_{x}}{D_{c}}$

As illustrated in FIG. 48, the catheter's end-point position can bepredicted given the joint or actuation coordinates by using the forwardkinematics equations described above.

Calculation of the catheter's actuated inputs as a function of end-pointposition, referred to as the inverse kinematics, can be performednumerically, using a nonlinear equation solver such as Newton-Raphson. Amore desirable approach, and the one used in this illustrativeembodiment, is to develop a closed-form solution which can be used tocalculate the required actuated inputs directly from the desiredend-point positions.

As with the forward kinematics, we separate the inverse kinematics intothe basic inverse kinematics, which relates joint coordinates to thetask coordinates, and the actuation inverse kinematics, which relatesthe actuation coordinates to the joint coordinates. The basic inversekinematics, relating the joint coordinates (□_(pitch), □_(pitch), L), tothe catheter task coordinates (Xc, Yc, Zc) is given as follows:

ϕ_(pitch) = α sin (θ) ϕ_(yaw) = α cos (θ)$L = \left. {R\;\alpha}\rightarrow\left. {where}\rightarrow\left. \rightarrow{\begin{matrix}\begin{matrix}\begin{matrix} \\\overset{\_}{\theta = {a\;\tan\; 2\left( {Z_{c},X_{c}} \right)}}\end{matrix} \\{R = \frac{l\;\sin\;\beta}{\sin\; 2\;\beta}}\end{matrix} \\{\alpha = {\pi - {2\;\beta}}}\end{matrix}\begin{matrix}\begin{matrix}\begin{matrix} \\\overset{\_}{\beta = {a\;\tan\; 2\left( {Y_{c},W_{c}} \right)}}\end{matrix} \\{\left. \rightarrow W_{c} \right. = \left( {X_{c}^{2} + Z_{c}^{2}} \right)^{1\text{/}2}}\end{matrix} \\\underset{\_}{l = \left( {W_{c}^{2} + Y_{c}^{2}} \right)^{1\text{/}2}}\end{matrix}} \right. \right. \right.$

The actuator inverse kinematics, relating the actuator coordinates(□L_(x),□L_(z),L) to the joint coordinates (□_(pitch), □_(pitch), L) isgiven as follows:

${\Delta\; L_{x}} = \frac{D_{c}\phi_{yaw}}{2}$${\Delta\; L_{z}} = \frac{D_{c}\phi_{pitch}}{2}$

Referring back to FIG. 42, pitch, yaw, and extension commands are passedfrom the inverse kinematics (450) to a position control block (448)along with measured localization data. FIG. 47 provides a more detailedview of the position control block (448). After measured XYZ positiondata comes in from the localization system, it goes through a inversekinematics block (464) to calculate the pitch, yaw, and extension theinstrument needs to have in order to travel to where it needs to be.Comparing (466) these values with filtered desired pitch, yaw, andextension data from the master input device, integral compensation isthen conducted with limits on pitch and yaw to integrate away the error.In this embodiment, the extension variable does not have the same limits(468), as do pitch and yaw (470). As will be apparent to those skilledin the art, having an integrator in a negative feedback loop forces theerror to zero. Desired pitch, yaw, and extension commands are nextpassed through a catheter workspace limitation (452), which may be afunction of the experimentally determined physical limits of theinstrument beyond which componentry may fail, deform undesirably, orperform unpredictably or undesirably. This workspace limitationessentially defines a volume similar to a cardioid-shaped volume aboutthe distal end of the instrument. Desired pitch, yaw, and extensioncommands, limited by the workspace limitation block, are then passed toa catheter roll correction block (454).

This functional block is depicted in further detail in FIG. 44, andessentially comprises a rotation matrix for transforming the pitch, yaw,and extension commands about the longitudinal, or “roll”, axis of theinstrument—to calibrate the control system for rotational deflection atthe distal tip of the catheter that may’ change the control elementsteering dynamics. For example, if a catheter has no rotationaldeflection, pulling on a control element located directly up at twelveo'clock should urge the distal tip of the instrument upward. If,however, the distal tip of the catheter has been rotationally deflectedby, say, ninety degrees clockwise, to get an upward response from thecatheter, it may be necessary to tension the control element that wasoriginally positioned at a nine o'clock position. The catheter rollcorrection schema depicted in FIG. 44 provides a means for using arotation matrix to make such a transformation, subject to a rollcorrection angle, such as the ninety degrees in the above example, whichis input, passed through a low pass filter, turned to radians, and putthrough rotation matrix calculations.

In one embodiment, the roll correction angle is determined throughexperimental experience with a particular instrument and path ofnavigation. In another embodiment, the roll correction angle may bedetermined experimentally in-situ using the accurate orientation dataavailable from the preferred localization systems. In other words, withsuch an embodiment, a command to, for example, bend straight up can beexecuted, and a localization system can be utilized to determine atwhich angle the defection actually went—to simply determine the in-situroll correction angle.

Referring briefly back to FIG. 42, roll corrected pitch and yawcommands, as well as unaffected extension commands, are output from theroll correction block (454) and may optionally be passed to aconventional velocity limitation block (456). Referring to FIG. 45,pitch and yaw commands are converted from radians to degrees, andautomatically controlled roll may enter the controls picture to completethe current desired position (472) from the last servo cycle. Velocityis calculated by comparing the desired position from the previous servocycle, as calculated with a conventional memory block (476) calculation,with that of the incoming commanded cycle. A conventional saturationblock (474) keeps the calculated velocity within specified values, andthe velocity-limited command (478) is converted back to radians andpassed to a tension control block (458).

Tension within control elements may be managed depending upon theparticular instrument embodiment, as described above in reference to thevarious instrument embodiments and tension control mechanisms. As anexample, FIG. 46 depicts a pre-tensioning block (480) with which a givencontrol element tension is ramped to a present value. An adjustment isthen added to the original pre-tensioning based upon a preferablyexperimentally-tuned matrix pertinent to variables, such as the failurelimits of the instrument construct and the incoming velocity-limitedpitch, yaw, extension, and roll commands. This adjusted value is thenadded (482) to the original signal for output, via gear ratioadjustment, to calculate desired motor rotation commands for the variousmotors involved with the instrument movement. In this embodiment,extension, roll, and sheath instrument actuation (484) have nopre-tensioning algorithms associated with their control. The output isthen complete from the master following mode functionality, and thisoutput is passed to the primary servo loop (436).

Referring back to FIG. 37, incoming desired motor rotation commands fromeither the master following mode (442), auto home mode (440), or idlemode (438) in the depicted embodiment are fed into a motor servo block(444), which is depicted in greater detail in FIGS. 38-41.

Referring to FIG. 38, incoming measured motor rotation data from digitalencoders and incoming desired motor rotation commands are filtered usingconventional quantization noise filtration at frequencies selected foreach of the incoming data streams to reduce noise while not adding unduedelays which may affect the stability of the control system. As shown inFIGS. 40 and 41, conventional quantization filtration is utilized on themeasured motor rotation signals at about 200 hertz in this embodiment,and on the desired motor rotation command at about 15 hertz. Thedifference (488) between the quantization filtered values forms theposition error which may be passed through a lead filter, the functionalequivalent of a proportional derivative (“PD”)+low pass filter. Inanother embodiment, conventional PID, lead/lag, or state spacerepresentation filter may be utilized. The lead filter of the depictedembodiment is shown in further detail in FIG. 39.

In particular, the lead filter embodiment in FIG. 39 comprises a varietyof constants selected to tune the system to achieve desired performance.The depicted filter addresses the needs of one embodiment of a 4-controlelement guide catheter instrument with independent control of each offour control element interface assemblies for .+−.pitch and .+−.yaw, andseparate roll and extension control. As demonstrated in the depictedembodiment, insertion and roll have different inertia and dynamics asopposed to pitch and yaw controls, and the constants selected to tunethem is different. The filter constants may be theoretically calculatedusing conventional techniques and tuned by experimental techniques, orwholly determined by experimental techniques, such as setting theconstants to give a sixty degree or more phase margin for stability andspeed of response, a conventional phase margin value for medical controlsystems.

In an embodiment where a tuned master following mode is paired with atuned primary servo loop, an instrument and instrument driver, such asthose described above, may be “driven” accurately in three-dimensionswith a remotely located master input device. Other preferred embodimentsincorporate related functionalities, such as haptic feedback to theoperator, active tensioning with a split carriage instrument driver,navigation utilizing direct visualization and/or tissue models acquiredin-situ and tissue contact sensing, and enhanced navigation logic.

Referring to FIG. 51, in one embodiment, the master input device may bea haptic master input device, such as those available from SensibleDevices, Inc., under the trade name Phantom™, and the hardware andsoftware required for operating such a device may at least partiallyreside on the master computer. The master XYZ positions measured fromthe master joint rotations and forward kinematics are generally passedto the master computer via a parallel port or similar link and maysubsequently be passed to a control and instrument driver computer. Withsuch an embodiment, an internal servo loop for the Phantom™ generallyruns at a much higher frequency in the range of 1,000 Hz, or greater, toaccurately create forces and torques at the joints of the master.

Referring to FIG. 52, a sample flowchart of a series of operationsleading from a position vector applied at the master input device to ahaptic signal applied back at the operator is depicted. A vector (344)associated with a master input device move by an operator may betransformed into an instrument coordinate system, and in particular to acatheter instrument tip coordinate system, using a simple matrixtransformation (345). The transformed vector (346) may then be scaled(347) per the preferences of the operator, to produce ascaled-transformed vector (348). The scaled-transformed vector (348) maybe sent to both the control and instrument driver computer (422)preferably via a serial wired connection, and to the master computer fora catheter workspace check (349) and any associated vector modification(350). this is followed by a feedback constant multiplication (351)chosen to produce preferred levels of feedback, such as force, in orderto produce a desired force vector (352), and an inverse transform (353)back to the master input device coordinate system for associated hapticsignaling to the operator in that coordinate system (354).

A conventional Jacobian may be utilized to convert a desired forcevector (352) to torques desirably applied at the various motorscomprising the master input device, to give the operator a desiredsignal pattern at the master input device. Given this embodiment of asuitable signal and execution pathway, feedback to the operator in theform of haptics, or touch sensations, may be utilized in various ways toprovide added safety and instinctiveness to the navigation features ofthe system, as discussed in further detail below.

FIG. 53 is a system block diagram including haptics capability. As shownin summary form in FIG. 53, encoder positions on the master inputdevice, changing in response to motion at the master input device, aremeasured (355), sent through forward kinematics calculations (356)pertinent to the master input device to get XYZ spatial positions of thedevice in the master input device coordinate system (357), thentransformed (358) to switch into the catheter coordinate system and(perhaps) transform for visualization orientation and preferred controlsorientation, to facilitate “instinctive driving.”

The transformed desired instrument position (359) may then be sent downone or more controls pathways to, for example, provide haptic feedback(360) regarding workspace boundaries or navigation issues, and provide acatheter instrument position control loop (361) with requisite catheterdesired position values, as transformed utilizing inverse kinematicsrelationships for the particular instrument (362) into yaw, pitch, andextension, or “insertion”, terms (363) pertinent to operating theparticular catheter instrument with open or closed loop control.

As discussed above, a system that controls a shapeable instrument can beimproved using a shape measurement. The shape measurement relies upon alocalization system as described above. In most cases the localizationsystem generates a plurality of data defining real-time or nearreal-time positional information, such as X-Y-Z coordinates in aCartesian coordinate system, orientation information relative to a givencoordinate axis or system. Typically, the reference of the coordinatesystem can be taken from one or more points along the shapeableinstrument. In additional variations, the reference of the coordinatesystem can be taken from one or more points on the anatomy, on therobotic control system, and/or on any other point as required by theparticular application. As noted herein, the methods, systems, anddevice described herein are useful for device shape sensing that usesthe shape data to improved catheter control, estimation, visualization,and diagnosis.

FIG. 54 shows a diagram of where shape information can be integratedinto one example of a robotic control topology. As shown, applicationsfor shape sensing can be categorized into three general groups basedupon where the shape information is fed into the control algorithm. In afirst example, the shape information is fed back into the cathetercontrol algorithms in order to achieve improved catheter control.However, as shown, the shape information can also be fed to a virtualenvironment or to estimate a position of the catheter.

FIG. 55A shows an example of a catheter control topology similar to thatshown in FIG. 1B however, in this example, the control topology isaugmented by shape information at several possible locations. As notedabove, the shape information (528) can be used for tip positionestimation (530), adaptive kinematic modeling (532), and/or catheterparameter estimation (534). There are numerous ways in which thisadditional data can be used to close feedback loops that do not relyupon the human operator. It should be noted that there are multiplediscrete points along the control algorithm in which shape informationcan be used to improve the robotic system.

Shape sensing is one observation into the state of a flexible device.FIG. 55B shows a basic control topology. Without measured shapeinformation, important positions along the flexible device must beestimated and controlled using other information. In its most generalform, shape sensing is an observer as shown in FIG. 55B. In thiscontext, observer refers to a control element that collects and usessensor data from the plant to provide contextualized information to thecontroller and/or the user.

The following disclosure includes combining different pieces ofinformation (e.g. position) with shape to estimate other unknowns (e.g.tissue contact). Shape is a very important piece of information for aflexible robot. Also, shape value increases when combined with otherinformation channels. For example, shape combined with a solid mechanicsmodel of the flexible section can be used to calculate possible forcesacting on the device. More simply, with the position of the base and theshape, the position of the tip is known. This is described furtherbelow.

FIG. 55C provides an example of information needed (in columns) toestimate other elements of the set (in the rows). Tip position refers toa position registered to an external reference frame. An internalreference frame in which a device is actuated is assumed.

For simple flexible devices, shape can be estimated, albeit correctlyonly in near ideal scenarios, with tip position and orientation, baseposition and orientation and a model of the device. To effectivelyestimate shape, position and orientation information is needed atmultiple points in the device. Corresponding to the complexity ofestimating shape, it is useful for estimating other things. If thedevice should follow a perfect arc (a simple model), knowing the tip andbase orientations and the path length (a subset of full positioninformation) is sufficient information to estimate shape from simplegeometric principles.

Tissue contact is useful for constructing geometric models of theenvironment or safely traversing constrained volumes. Contact can beinferred from shape deflection from the intended shape. However, analogforces may also be inferred from this difference in shape. We would liketo simplify the estimate requirements to the most straightforward orsmallest subset. Thus, we can say tip contact can be estimated fromdistal forces (which might be measured with local strain gauges or alsoinferred from IntelliSense measurements). If a three-dimensional modelalready exists, measured position registered to the model shouldindicate whether the device is in contact near the measurement.

If shape base position and base orientation are known, tip position andorientation are straightforward to calculate. Considering that tipposition and orientation may be measured directly with off-the-shelfproducts (NavX, Carto), it may be more useful to consider the case withshape and tip position and orientation measurements. Base position andorientation can be calculated with shape and tip information. Further,position and orientation at any point along the path of the device arealso known. With a geometric model of the environment, knowledge of theentire device position can be used to prevent or reduce contact or planpaths.

Since a device model (along with known actuator inputs) should allowestimation of ideal device shape, real shape could be used to adapt themodel to achieve closer agreement between expected and measured shape.Device contact with the environment is important because it will causeits own shape deflection and thus the model adaptation could ignore datawhen in contact or include contact forces in the estimation (asdiscussed further below).

Distal forces can be estimated from shape, a solid mechanics devicemodel and knowledge of contact. If the device is inserted straight butthe measured shape indicates a bend, the amount and point of applicationof force must be estimated. The device will bend differently if force isapplied at the middle of the bisected length than if force is applied atthe tip. The shape of the bends along with their degree and the solidmechanics model will indicate external forces applied to the device.More detailed descriptions of the estimation of force and the use offorce data will be described below.

Knowledge of contact can be used to create a model of the environment inthe internal reference frame. Position information registered to anexternal reference frame allows such a model to also be registered.Shape is not indicated in the table for estimating a geometricenvironment model, but shape is important for estimating contact.

Finally, positioning element tension (and other internal device forces)can be estimated using shape and the solid-mechanics model of the device(assuming no contact). Positioning element tension may also be used inestimating or improving estimates of other items of this set, but shapeis more important or useful.

Task Space Feedback:

A task space application involves those situations where a roboticsystem attempts to position a working end of a shapeable device at atarget region or goal. Applying shape feedback information to a taskspaced application can assist in producing the desired task spacepositions and motions.

The shape of a catheter, when given some reference, can be integrated toyield position or orientation, such as tip position estimation (530)represented in FIG. 55A. There are multiple other methods for measuringpositions and orientations for points along the length of a catheter.Points of particular interest can be the termination of positioningelements on a control ring, termination of pressure vessels or any otheractuator, or transitions in catheter stiffness. FIG. 56A represents ashapeable instrument (70) when navigated through an environment. In sucha case, some points of interest might include intermediate points (72)along a lengthy section where inflections in curvature are likely tooccur and the resulting effects on the tip portion or points (74)adjacent to the tip. The location of these points can be determinedexperimentally by observing shapeable instrument or a similar model inthe relevant anatomy that can indicate appropriate sampling frequency tocapture the observed bending state. Alternatively, modeling of theshapeable instruments mechanics can be used to determine those pointsthat should be measured.

The task space can be specified in terms of the distal tip motion;however a given task may be concerned with intermediate proximal pointsas well to traverse a path. Therefore, task space control can includethe explicit control of one or more positions and orientations along thelength of the flexible device. In the simplest case, the model assumesfree motion of all sections (such as in an open heart chamber) where thesystem controls the distal tip position to apply some therapy such asablation. In this case, the system feeds an estimated tip position andorientation and compares against an input reference position as seen inFIG. 56B.

This error signal can then go through some compensator (550) such as atime derivative and gain to yield a command such as a tip velocity. Theforward kinematics can then be inverted differentially (552), (e.g., viaJacobian pseudo-inverse or other non-linear constrained optimizationtechniques) that translates the velocity command into an actuatorcommand (such as displacement or tensioning of a positioning element).These actuator commands in turn put a force on the instrument and as itinteracts with the environment (captured together as plant (554)), thesensor will read the new position/orientation for further feedback. Ifthe sensor were not available, the feedback could simply be the modelforward kinematics (556) that would at least prevent integration errorin this scheme.

One example of task space feedback control using the scheme shown inFIG. 56B is to control an automated ablation inside the heart. In suchan example, a 3-D geometrical model of the heart can be used incombination with the known location of the instrument (in this case acatheter) in the model to carry out a circular ablation around thepulmonary veins. A user could define points around the pulmonary vein.The system would then calculate a spline in between these defined pointsto further define the path of the catheter. The real catheter positionwould then be measured, and the difference between this position and theinitial ablation point would be the error signal sent into thecompensator (550). The inverse kinematics (552) then transforms thedistal tip position error into commands that drive actuators to adjustpositioning elements in the instrument to produce desired displacements.In some cases, the system can optimize for low force in the positioningelements. The catheter tip would then move towards the initial ablationposition and the real position would be measured once again. Once thecatheter is detected to be within some threshold of this initialablation position (i.e., the error signal is sufficiently low), thecommanded position would then begin to move along the ablation path. Theuser could adjust the speed of motion, as well as the position errorthreshold by which the catheter should stop moving if it exceeds.

The preceding example only uses the measured tip position as a means ofgenerating an error signal. However, the sensor feedback can be used forother purposes. For example, the Jacobian (or inverse kinematics) can beadaptive based on shape or angle, or other feedback as describedpreviously. This can be important because the inverse kinematicsthemselves assume a known configuration of the catheter and updatedinformation would be beneficial. The adaptive component could be assimple as updating configuration parameters such as angle and curvature,or could be as complex as a learning algorithm that adapts over time tolearn the mapping from commanded to achieved velocity.

An even simpler form of distal tip feedback control is simply to run thenormal inverse kinematics as in FIG. 55A but with the tip positionestimation used to feed in an error signal. In this case, the originalposition command into the inverse kinematics will now be an error signalwith a gain term (and potentially integration, differentiation, or otheroperations), which should move the catheter in the desired direction.This method of operation could also benefit from adaptive kinematics toaid with directionalities. It could also benefit from lower levelconfiguration feedback control as described previously so the assumedshape is more likely to be correct.

These methods for controlling the distal tip position and orientationcould also be applied to more intermediate points of interest providedsufficient degrees of freedom. In some variations, it may be necessaryto specify weightings or some other criteria on specific task goals ifthe dimension of the task space is greater than the dimension of theactuator space. In variations where the system controls multiple pointssimultaneously, the system might select directions that align axiallywith the local point so as to move along the present path.Alternatively, the system can place emphasis to distal degrees offreedom since proximal motions have large effects on distal points dueto the moment arm from proximal constraints. These trade-offs motivatelower level control of the device in configuration space.

Configuration Space Feedback:

Shape can describe the configuration of device better than feedback of asingle position or orientation. Having the capability to obtain shapeinformation allows for the current shape of a device (including angle,curvature, profile, torsion, etc.) to be fed back into a controller. Thecontroller can then process a command to actuate the device into adesired shape. FIG. 57A illustrates one example of a modification toapply shape feedback information into an existing closed loop system toalter a position control signal. As shown, shape sensing provides ameasured configuration (via localization data) of a real shape of theshapeable device. This measured configuration is then compared to adesired configuration. Where the desired configuration is either modeleddata or a known ideal configuration such that a differential between thereal and desired shape can be quantified. Here, the feedback system usesthe difference between the measured and desired catheter configurationto produce a configuration error, error signal, or signal that is thenapplied to a feed-back controller (500) to modify the configurationcommand sent to a feed forward control element that effects a responsein the shapeable instrument or catheter (502). The difference betweenthe measured and desired configuration can be applied through a gainelement (504) as shown. The gain element can include a multitude ofcontrol elements such as proportional, derivative, or integral terms.This illustrated configuration uses shape data to alter a feed forwardcommand that drives the shapeable instrument to a desired target.

FIG. 57B illustrates an alternative closed loop control configurationfor a pure feedback control form that uses an error between the measuredor real shape data and the desired shape. This error is then fed as aninput to the feed back controller (506) to generate a feed back signalto control the shapeable instrument (502). This configuration does notdepend upon a model-based feed forward control element (sometimesreferred to as model-based control). However, a pure feedback controlleras shown here may require integrator terms internally in order toachieve steady-state error requirements.

Because both pure feed forward and pure feedback topologies have clearlyidentifiable advantages and disadvantages, the topologies can becombined as shown in FIG. 57C to realize the benefits of both feedforward and pure feedback configurations. As shown, the feed forwardcontrol element (500) uses its detailed model knowledge to improvetransient tracking and to reduce dependence upon large integrator terms.The feedback control element (506) can help reject environmentaldisturbances or modeling errors by modifying the actuator commandscoming from the (hopefully dominant) feed forward control element (506).

FIG. 57D shows another variation of a control system using an integratedfeed back and feed forward controller. This allows the feed backcontroller to exercise control authority at one or more points withinthe feed forward controller.

FIG. 57E shows one example of an integrated feedback and feed forwardcontroller. As shown, in the integrated controller (508), feedback termsare inserted into the existing (model based) feed forward controlalgorithms. In this example, configuration error, once mapped throughappropriate gain elements (509), (510), (511), can be injected asfeedback terms into the existing model-control pathway at severaldiscrete locations. In the illustrated example, the configuration error,is used to modify the moments and forces coming from the beam mechanicsmodel, the individual tendon tension commands, or the final tendondisplacement commands. However, any of these methods can also apply toalternative actuation methods such as remote micro actuators (voltagecommand), fluid channels (pressure command), thrusters (ejection speed),magnets (orientation or shield duty cycle), etc.

Tracking with Shape Information:

In an additional variation, systems, methods and devices using shapeinformation can be used to track advancement of a shapeable instrumentthrough an anatomic path. The anatomic path can include a path through avessel, organ, or between organs. For example, the anatomic path caninclude a path through one or more vessels to access the heart.Alternatively, the path can include navigation through bronchialpassages or the digestive tract. The robotic system then monitors theshape of the instrument for any changes in shape that would indicate theneed for a corrective action.

FIG. 58A illustrates an example of endovascular tracking of a shapeableinstrument. In this example, the shapeable instrument (50) is advancedthrough an iliac artery (100) using a robotic system (52) that monitorsshape information to adjust advancement of the instrument (50). FIG. 58Aillustrates a desired shape (54) of the instrument (50) when advancedinto the iliac artery (100) and across the aorta (102). As shown, if theshapeable instrument (50) advances as desired it will naturally assumethe shape of the anatomy. During insertion, it is desired that thedistal tip of the catheter advance in the direction it is facing.However, as shown in FIG. 58B, if the instrument (50) fails to advancein the desired path, the instrument (50) assumes a shape (54) thatvaries from the desired shape (52). In this example, the instrument (50)assumes a sharp bend as it backs into the aorta due to the lack of anyconstraining anatomy. In this situation, the distal tip of theinstrument (50) either ceases advancing, or even backs further into theaorta. This motion in the opposite direction of intended could be amajor problem for a robotic system (52) that is supposed to beintuitive.

To protect against this situation, the robotic system (52) monitors theshape of one or more portions of the instrument (50) to detect for theexpected shape or to monitor for unexpected shapes such as the sharpbend (56) shown in FIG. 58B. In some variations, the robotic systemsaves a present or natural shape of the instrument (50) as a referenceindicative of the shape of that anatomy. The robotic system (52) couldeven save a shape of the anatomy as a reference. During insertion, theprogressing instrument (50) should approximately follow the reference iftracking is occurring successfully. If a sufficiently large deviation isdetected in the present instrument (50) shape (56) with the reference ora desired shape (54), the robotic system (52) ceases instrument (50)insertion to prevent backup into the aorta. In an additional variation,using knowledge of the location of the deviation, the instrument (50)can be actuated near that location to attempt to break friction with thewall and descend in the direction of intended tracking. Alternatively,the instrument (50) can be withdrawn and advanced in a different mannerto prevent the backup. This procedure, or similar actuation scheme, maybe repeated along with continued insertion to resume tracking.Alternatively, if the shape detects tracking failure, a guide (58) couldbe inserted out of the distal tip of the instrument (50) as far aspossible to obtain purchase (or stability). This guide (58) could thenbe left in the anatomy to provide a track for continued tracking of thecatheter.

Although tracking of shape is beneficial when advanced using a roboticsystem, tracking of shape can also benefit procedures in which theinstrument is manually advanced or advancement is assisted via a roboticsystem. In such cases, the instrument will be coupled to a system thatcan provide information to the operator regarding the shape of theinstrument as described above.

Reduced Model Control:

One of the primary benefits of feedback control is avoiding reliance onmodels and their associated errors. The less data that is available, themore important a model becomes. With full state feedback of somecombination of shape, position, orientation, or deformation over asufficient history, the relationships between actuator and position orshape can be learned. For example, during a procedure the robotic systemcan use shape data to correlate a map between force on one or morepositioning elements and multi-section bending of the instrument. Thismap can be inferred from many measurements. For a particular region,this map may depend on the anatomical constraints and may need to berefreshed (or continuously updated) if conditions are significantlyaltered. To supplement the data, a model could be used for the anatomyto estimate its presence and characteristics as it interacts with thecatheter. This mapping could at one extreme serve as a black box withinthe controller (508) in FIG. 57D where the user input is first relatedto some description of the state of the shapeable element and themapping is used to find the actuator commands. In this example, there isvery little modeling, mostly on the high-level user interface relatingthe user's intentions to instrument state. Otherwise, the sensor datacan be used in place of a model and continuously updated.

A simple use of reduced model control could be to monitor the mappingbetween proximal instrument insertion and distal tip motion. A distaltip of the instrument can move in almost any direction under proximalinsertion depending on distal contact with anatomy. Therefore, knowingthis mapping between proximal insert and distal motion is of criticalimportance in maintaining intuitive robotic control. In the simplestform, the user could request a calibration where the proximal insertionwould be dithered, and the distal tip motion measured. Subsequentmovements of the master input device in the measured direction wouldthen map to distal insertion and preserve intuitive driving.

To further reduce the model, it might be possible to instrument anon-robotic catheter with shape sensing for a given procedure orsub-routine of a procedure. The specific goals of each part of theprocedure could be stated and associated with the measured catheterstate. This association could provide the basis for a mapping betweenuser intent and catheter result that could be applied to a roboticsystem. The user could then simply operate the robotic system insupervisory fashion where the user provides a command such as cannulatethe carotid and this command is then translated to catheter shape, thenactuator commands: This process could be largely based on ck1ta historyand not be overly reliant on complex models.

Different device and control architectures, shape sensing accuracies,and device configurations requires different modeling trade-offs. A lessaccurate shape sensor may require more device modeling to achieveaccurate control. Thus, models are useful in many scenarios. Kinematicmodels of robotic devices are very useful. The strong geometric basis ofkinematic models is well informed by shape. As discussed below, systemsand methods use shape information to adjust or inform kinematic models.

Applying Shape Information to Kinematic Models:

Due in large part to the open-loop nature of the existing instrumentcontrol, the instrument mechanics algorithms can be dependent upon anaccurate kinematic model that represents the mechanical and physicalcharacteristics and parameters of the instrument. Some of the parametersin this model are “tuned” at development/design time while others arecharacterized on an individual device basis or lot-by-lot basis as partof the manufacturing process. A partial list of these model parametersincludes: segment length, overall lengths, diameter, bending stiffness,axial stiffnesses, and control positioning element stiffness, etc.

In one variation, systems and methods disclosed herein can use aninstruments measured shape data in combination with commanded shape,commanded tendon displacements, measured tendon displacements, and anyother available sensor data (e.g. measured tendon tensions) to estimateimproved parameters for the instrument model used by the instrumentmechanics algorithms. This combination can modify an instrument'sconfiguration control algorithms as discussed below. Alternatively or incombination, the combination can allow for new parameter values thatmake the control model more accurately match the specific instrumentbeing manipulated. The estimation of these improved parameters can beaccomplished with existing model-fitting, machine learning, or adaptivecontrol techniques. FIG. 59A represents an example control relationshipwhere shape sensing occurs after the real instrument is positioned. Theresulting shape sensing data is fed to an estimation of the catheterparameter along with virtual catheter configuration as well as virtualand actual tendon displacement. The resulting error or difference can beused to update the catheter mechanics in the catheter model forimproving future commands generated by the robot control system.

Another application of applying shape sensing data involves improving aperformance of the instrument driving when a mismatch occurs between anassumed and a real kinematic relationship. For example, as shown in FIG.59B, when measuring a real or actual shape of an instrument, the controlsystem can adapt the inverse kinematic algorithms, if necessary, to moreclosely match how the real catheter actually moves based on the errorbetween the desired instrument configuration and the obtained shapedata. Thus, the inverse kinematic model becomes updated or improvedbased on real correlation of the movement of the instrument. This allowsfor the robotic control system to generate commands an improved inversekinematic model that contains a more realistic set of inverse kinematicrelationships. As a result, an actual tip position of the instrumentshould more closely track a desired tip position.

In a general implementation, the adaptive kinematics module will likelycombine measured shape information with knowledge about the commandedshape, actuator commands, and any other sensory information that may beavailable (e.g. measured pullwire or positioning element tensions).Several more specific algorithmic implementations are listed below.

One variation of an adaptation scheme includes a “trial and error” typeapproach. This approach requires an initial guess as to instrumentposition with a subsequent collection of the shape data to determine theactual result. The control system then compares the initial guessedposition or shape with the actual measured resulting shape and uses themeasured error signal to improve correlation to learn from the error. Inthis sense, the control system uses an original or idealized inversekinematic model to compute the initial robot control commands wheneverthe user starts driving in a new portion of the anatomy or driving in apreviously unexplored direction. Because of this, the instrumentsresponse to the user's first motion commands should contain the greatesterror. As the user makes repeated efforts to access a target, theresponse of the control system to manipulate the instrument shouldincrease in accuracy.

In another variation, an adaption scheme could include the use ofbuilding a lookup table of commanded catheter configurations vs.actually achieved catheter shapes and their corresponding tip positionsrelative to the base coordinate frame.

In one example, adapting a kinematic model can benefit on a temporarybasis. For example, if the shapeable instrument encounters an anatomicconstraint during navigation the behavior of the instrument will beaffected as long as it encounters the constraint. For instance, a commonoccurrence is where a shapeable instrument comprises a catheter where aproximal portion of the catheter is largely constrained by a bloodvessel and only a distal portion of the catheter is free to articulate.In this case, adapting the kinematic model can be as simple asestimating a new effective articulation length based upon sorting thelength of the catheter into a section that, upon measuring shape, isobserved to move freely and a section that generally is not changingshape (or encounters significant resistive force when changing shape).

In another example of adapting a kinematic model, a measured shape of aninstrument can be used to compute curvature as a function of arc lengthalong the instrument. By analyzing the shape data for sharp transitionsor discontinuities the assumed articulating length of the cathetercan˜be broken up into several subsections that may be behavingdifferently from each other, but within each subsection the behaviormore accurately matches the assumed constant curvature arcs. Based uponthis auto-segmentation, the inverse kinematics can be computed as achain of several smaller subsections that each can be accuratelycomputed with the traditional kinematic models.

In another variation, shape data can be overlayed, as shown in FIG. 60.As illustrated, the shapeable instrument (1) is articulated orrepositioned (either automatically by the robotic control system ormanually by an operator). In doing so, the instrument (1) could use athreshold resistance value so that when encountering the resistance, theinstrument moves to another position. Once a period of time passes, thesystem assess all of the shapes collected during the window, typicallyduring intervals. The system can then look for voids or regions wherethe instrument was forced to avoid. These void spaces will likelyrepresent some sort of environment constraint (obstruction) that can beincluded in the computation of the inverse kinematics. Once the systemidentifies these environmental obstacles, the system can apply a widerange of existing path planning algorithms to help solve for desiredinstrument shapes that achieve a desired tip position while avoiding theidentified obstacles. Furthermore, the information shown in FIG. 60 canbe provided in a graphical form for observation by the operator.

Improved Kinematic Models

An initial or existing inverse kinematic model is typically based upon aparametric model that describes the number and nature (length, degreesof freedom, coupling, etc.) of bending segments of the shapeableinstrument. One variation of improving the kinematic model is to adaptthe model via a model fitting exercise. The system or an. operator canengage the robotic system to perform a training routine in which theshapeable instrument moves through a series of locations and shapes.Once sufficient training data is collected on the observed correlationbetween instrument configuration (shape) and tip position, one or moremodel fitting techniques can find the parameter set for the kinematicmodel that produces predictions that best match observations or thatproduces the predictions desired for the particular application. Thisbest fit model is then used to compute the inverse kinematics for futureiterations.

In yet another variation of adapting the model, an implementation of anadaptive kinematics module could use some intelligent combination ofseveral of the concepts described herein. Keep in mind, that while allof these approaches deal with the translation of commands from taskspace to configuration space, they do not necessarily all employ theexact same description of the catheter in joint space.

Adaptive Jacobian

Another approach to adapting kinematics is to solve the inversekinematics based on velocity rather than position. Typically, at eachtime step the system solves for an instrument configuration thatachieves a desired tip position. Instead, the system can solve for thechange in shape of the instrument, relative to its current configurationor shape, where the change produces a desired incremental movement orvelocity of the instrument's tip change in catheter configurationmathematically, the general relationship between velocities is aderivative of the kinematic relationships between positions thisrelationship is referred to as the Jacobian. One significant benefit ofan adaptive Jacobian model is that exact kinematic relationships betweenpositions are not needed. The model could simply infer, estimate, orlearn the Jacobian from the sensed shape information. However, whencombined with other methods, determining the relationships would berequired.

One example of a very simple adaptive Jacobian approach is in a modifiedcontrol architecture where a configuration space command is a measure ofhow hard the system is trying to bend or direct the shapeable instrumentin a particular direction. Using a measurement of the shape of theinstrument and its associated tip orientation, the adaptive Jacobianapproach translates a tip velocity command into a configuration commandsas follows:

1) To move the tip laterally (relative to the tip orientation) in thedirection of the catheter bend, apply more bending effort to thecatheter segment.

2) To move the tip laterally (relative to the tip orientation) in thedirection opposite the catheter bend, apply less bending effort to thecatheter segment.

3) To move the tip in the direction it is pointed, insert the cathetersegment.

4) To move the tip in the opposite direction than it is pointed, retractthe catheter segment.

As noted above, one advantage to this control model is to reduceddependence upon accurately modeling all aspects of the catheter and theenvironment with which it interacts. The model simply relies on movingthe shapeable instrument and’ its tip from its current position to adesired position.

Real Shape Display

Estimation of an instrument's mechanical parameter can be performed postmanufacturing to be set statically for the procedure. Alternatively orin combination, mechanical and other parameters can be updatedintraoperatively to improve control. When the properties of a flexiblesection are used to determine control inputs, system feedback can beused to estimate those properties. As a basic example, the stiffness ofa section of the instrument can be used to decide how much to displacepositioning elements (e.g., pull tension wires) to produce a desiredconfiguration. If the bulk bending stiffness of a section is differentthan the existing stiffness parameters, then the section will not bendor bend too much and not produce the desired configuration. Using shapedata to assess the shape of the instrument, a new bending stiffness canbe continuously updated based on the degree to which the positioningelements are displaced and the resulting actual bending. Arecursive-least-squares technique can be used to quickly recalculate anew bending stiffness as new data is measured. However, any number ofother methods of adapting parameters can also be \!Sed. FIG. 61Aillustrates a general system block diagram where the goal of adaptationis to choose the characteristics of the model that minimize thedifference between y_(measured) and y_(predicted).

However, in adapting the actual instrument parameters, it will be usefulto determine if the instrument is in contact with the environment. Whenthe instrument is operated in free space, the parameters can be adaptedto reduce the difference between the desired configurations versus thepredicted configuration. However, if the instrument (1) is not bendingas far as expected due to contact with an external object (49), as shownin FIG. 61B, then updating the bending stiffness parameters should notbe updated unless the instrument (1) is to be operated while engagingthe object (49). Accordingly, this model will benefit from measuringexternal forces on the instrument (1) prior to altering mechanicalparameters.

One instrument state that is difficult to observe in practice is acompression or extension along a longitudinal axis of the shapeableinstrument. Knowledge of the compression or extension of the instrumentis important in the control of the instrument because the actuatorsoften act in series with this axial mode. For example, positioningelements or tendon actuation can be used to control an instrument byrouting the positioning elements through a conduit along a length of theinstrument. The control elements can be terminated at the distal end andactuated proximally to alter a shape of the instrument. In this case,compression of the instrument also affects the positioning elements andcould even lead to slack. Therefore, the axial compression ore extensionshould be known or estimated to maintain improved control authority. Inaddition, the axial deformation can be important if some other member isrouted co-axially with the shapeable instrument. For example, anablation catheter can be routed down a central through lumen. If theinstrument compresses, the ablation catheter becomes further exposedClinically, this exposure, or protrusion, can cause difficulty inperforming articulations in small spaces since the uncontrolled ablationcatheter will be sticking out further and occupying significant volume.

Ideally, an observer or a localization system could provide completedeformation information for the entire length of the device (similar toFinite Element Method results), including the axial deformation. Inreality, measurements only of select states are practical, eachpotentially requiring its own unique sensor. In the case of axialdeformation, there are a number of potential methods to sense thedesired information. For example in FIG. 62A, an optical fiber (12) canbe incorporated into a shapeable instrument (1) as described above.However, in this variation, one or more the optical fibers (12) areconfigured in a helix within the instrument (1). The helices of theoptical fiber or fibers (12) provide mechanical flexibility and canprovide for estimating compression of the instrument (1).

Alternatively, as shown in FIG. 62B a longitudinal multi-core fiber (12)could be allowed to float either along the centroid of the shapeableinstrument 0), or at some other distance from the centroid. At theproximal end (17) the fiber (12), compression or extension of theinstrument (1) causes the fiber to move relative to the instrument sinceit is free-floating. Axial deformation or expansion can be measured bymeasuring the movement of the fiber (17).

This linear motion of the fiber (17) can be measured by a linear encoderor similar method. Since the bending can be calculated using the BraggGratings or another localization system, the manipulator's axialcompression can be found from this proximal linear motion of the fiber.Additional variations of this configuration allow for any other flexibleelement to be used instead of the freely floating fiber. Such flexibleelements include, but are not limited to: a wire, coil tube, oractuation element so long as its own axial deformation is observable orreasonably assumed to be negligible. If such axial deformation ispredicted to be significant, the force on the elongate element could bemeasured to estimate its own axial deformation. Similarly, the force ofall actuation elements could be measured or modeled to estimate thetotal axial deformation of the flexible manipulator itself.

Given information of axial deformation, there are various steps that canbe taken to make use of this information. For example, as shown in FIG.62B, an ablation catheter could be controlled to axially adjust so thatthe amount exposed at the distal end of the shapeable instrument (I) iscontrolled. Similar to that shown in FIG. 62B, in an architecture withmultiple concentric manipulators (such as the Artisan inner and outerguides manufactured by Hansen Medical), inner and outer guide insertpositions can be adjusted to account for axial deformation as well. Infact, the axial deformation could be used for manipulation of thedevices to achieve a desired axial motion. This axial motion could beuseful for extending reach, dithering out tendon friction, ablationcatheter friction, or otherwise. Another use of active axial deformationcontrol would be to pre-load the manipulator by a quantity sufficient tohold constant compression over the range of articulations. Then theablation catheter would appear to stay fixed with respect to the guidecatheter.

As previously mentioned, the axial deformation can be important simplyfor maintaining control authority as it affects the actuators. Forexample, if the axial deformation were accurately known, that quantitycould be adjusted via the actuators that drive the positioning elementsin addition to the amount due to bending and deformation of thepositioning elements themselves. Furthermore, with accurate knowledge ofthe axial deformation combined with some knowledge of bending, theabsolute angle of a point on the catheter may be identifiable withrespect to the robot. For example, if the total force on the catheter ismeasured or modeled, we can infer the axial deformation based onstiffness.

An alternative way to measure the axial deformation would be to obtainlocalization data (from sensor, image processing, etc) of the ablationcatheter differenced from the inner guide catheter. Cumulative bendingand axial deformation can also be obtained by the amount which a coiltube (60) (which is axially rigid and allowed to float in the catheter)displaces out of the proximal end of the instrument (1). The coil tube(60) displacement less the axial deformation estimate yields pure bendinformation that indicates the angle of deflection A of the distal coiltube termination with respect to the robot as shown in FIG. 62C. Thisangle information could be considered part of the shape sensing feedbackwhere it can feedback to the user, high level control, or low levelcontrol. For instrument control as in FIG. 62C, the angular informationgleaned from axial deformation could be used for tip pose feedback, inan adaptive model by updating directionalities, or in catheter parameterestimation for updating stiffness and more.

Pre-Tensioning

When a shapeable instrument is coupled to a robotic control mechanismthere is typically an unknown amount of slack in positioning element. Ifthe catheter is to be effectively controlled by position (rather thantension) controlling each of the positioning elements, this unknownpositioning element slack must be removed by finding appropriateposition offsets for each positioning element. Pre-tensioning is theprocess of finding out how much slack is in each tendon and removing it.

Several sources of initial slack in the tendons are illustrated in FIGS.63A to 63C. As shown in FIG. 63A, the lengths of positioning elements(62) can inconsistent due to manufacturing tolerances. Additionally, asshown in FIGS. 63B and 63C, any bends in the proximal (non-articulating)or distal (articulating) portions of the shapeable instrument (1) willalso contribute to an offset in the position of the positioning elements(62) due to different path lengths along the inside and outside of thebends. One way in which we deal with is to slowly drive the articulationaxes until the operator “feels” (by monitoring motor currents or sensedwire tensions) the positioning elements (62) pull taught. Without shapesensing, this procedure is dependent upon two critical assumptions: 1)The articulating portion of the catheter is straight when pre-tensioningoccurs; and 2) The non-articulating portion of the catheter does notchange shape after pre-tensioning occurs.

The first assumption is important because by pre-tensioning is intendedto find the “zero-point” for the control element displacements. It isfrom this “zero point” that control element commands are added todisplace the control element to control the shapeable device. If thearticulating portion of the catheter is in fact bent duringpre-tensioning, this will introduce an un-modeled disturbance to thecatheter control algorithms and result in degraded tracking of usercommands. In theory, this assumption could be relaxed to require thatthe articulating portion of the catheter is in any known configurationduring pre-tensioning. However, without shape sensing, a straightcatheter (held there by the catheter's internal bending stiffness) isthe most practical configuration to assume.

The second assumption comes from the tact that any motion in thenon-articulating portion of the instrument (1) introduces additionaloffsets to the positions of the positioning elements (62). Because theconfiguration of the non-articulating portion of the instrument (1) isneither modeled nor (without shape sensing) sensed, the instrument (1)control algorithms are unaware of these changes in the positioningelement (62) displacements, therefore resulting in degraded tracking ofuser commands. Conventionally, pre-tensioning the instrument (1) occursafter it has been inserted into the patient and taken the shape of thepatient's anatomy. If subsequent significant changes to the shape of thenon-articulating portion of the instrument (1) occur, the operator muststraighten out the articulating portion and re-execute pre-tensioning toaccount for these changes.

Shape sensing provides an opportunity to relax both of theseassumptions. The articulating portion of the instrument (1) could bepre-tensioned in any configuration by using the shape sensing system tomeasure the configuration of the catheter at the completion ofpre-tensioning. The measured pretension offsets are then corrected bycomputing the deviation in position of the positioning element (62) dueto the measured configuration. This computation itself if very similarto portions of the existing instrument (1) control algorithms.

The non-articulating portion of the instrument (1) can also be allowedto move after pre-tensioning by continually or periodically measuringthe shape of the non-articulating portion. The pretension offsets canthen be corrected by computing the deviation in tendon position due tothe measured change in the shape of the non-articulating portion of theinstrument (1).

Reaction Force

While contact deflection can disturb catheter parameter adaptation, useof shape data allows contact deflection to be used to estimate thecontact force. If the bending stiffness of a section is known or hasbeen adapted before coming in contact with an obstruction (49), thedeviation between a predicted shape and an actual shape as measured canreveal where the instrument (1) makes contact and the degree of forcewith the environment or obstruction (49). If the estimate of bendingstiffness, calculated simply from tendon positions and shape, were torapidly change in conjunction with commanded movement, the device couldbe assumed to have come into contact with its environment. The newbending stiffness should be ignored in lieu of that calculated prior tothe rapid change. Next, since the shape is dependent on the controlinputs, device properties and the force from the environment. The forcefrom the environment can be determined or estimated since the controlinputs and device properties are known. For example, as shown in FIG.61B, when the shapeable instrument (1) encounters an obstruction (49),the change in shape is measured and the control inputs as well as thedevice properties are know. Therefore, the system can calculate theforce required to deflect the instrument (1) into its real position.

Estimating Force Applied to Anatomy with Shape Information

With no interaction with the environment, a flexible device will have adefault shape for a sequence of actuator inputs. If the flexible devicecontacts with the environment, the device (or a portion) will deflect inshape along portions of the device again as shown in FIG. 61B. A correctestimation of contact forces applied against the shapeable instrument ina solid-mechanics model can minimize the difference between theestimated shape and measured shape that results from contact. However,there can be multiple combinations forces that will achieve thesolution. The reaction force can be presumed to occur at specificlocations based on the expected shape of the device. The device willoften contact the environment at the tip with a reaction forceperpendicular to the tip. The device may also contact along the body. Tosolve for body contact, a force can be applied to a beam model of thedevice at various locations along the length of the device model. Theresulting deflected model that best matches the measured shape can beselected as best describing the reaction force.

Also, if there are multiple possible shapes for a set inputconfigurations (i.e. path dependence), the previous configuration mayalso need to be considered in solving the forces.

If shape is not tracking the command, it could be in contact withtissue. Constantly observing the error between commanded shape andactual shape allows rapid detection of contact. An obvious metric forthis error is the norm of the distance between measured and commandedpositions along the path of the catheter (any number of norms could beused). However, this metric would amplify disturbances in curvature atone point (since such a disturbance is integrated to position). Anothermetric includes the mean square error between commanded and measuredcurvature at each point. This weights the points more evenly andattributes error properly at the afflicted location rather than theeffect of the error on other points, as distance does.

Once contact is made, there are several possible actions depending onthe severity of tissue contact. If no tissue contact is tolerable, thecontroller generates a signal or stops the instrument from moving and analarm notice may be posted for the operator. In another variation, thecontrol system may issue commands to automatically retrace motion tomove away from the contact by a set distance.

If contact should merely be limited, there are more possibilities. Forexample, haptic feedback such as a vibration or force can be applied tothe input device to indicate contact. Vibration is useful if thelocation or direction of contact is unknown. However, a reasonable guessof direction is the vector of motion when contact occurred. Force couldbe added to resist motion in that direction on the input device. Also,slave movement could be de-scaled when in contact.

If force is estimated as previously described, a haptic force could beapplied to the input device proportional to the estimated force. Thedirection of the master haptic force should be that which will reducethe reaction force in the slave.

Use of Force Data for Path Planning

If the environment geometry is known, the forces can be estimated forall actual instrument configurations. Two sub-cases are described wherethe goal is either known or unknown.

If a goal position in the environment is known, the path to that pointcan be planned given knowledge of the environment geometry. The bestpath will depend on the cost associated with forces applied to theinstrument (1). For example, one example constraint could be that distaltip forces must be below one threshold and body forces must be belowanother (this could be to effectively limit the pressure on tissue).These two conditions constrain the possible path to the goal. In fact,there may be no path and this could be communicated to the operatorallowing the operator to change threshold or reposition the startingpoint. If there is one path that meets the constraints, it is likelythere are many Haptic forces or visual shading may be used to guide theoperator to stay in the set of acceptable paths. Also, the path whichminimizes peak forces (or sum-of-square forces) may be calculated andlight haptic forces could be used to guide the operator to drivedirectly along that path. Similarly, that path may be automaticallydriven by the robot.

If the goal is unknown, there may still be a predictable set ofacceptable paths through the environment. If the operator is interestedin two distinct modes—maneuvering to a specific area then applyingtreatment in that area, the user interface can provide an interface toswitch between the two modes. In the maneuvering mode, the path plannerwill try to keep the treatment tip and catheter body away from tissueand minimize forces along potential paths. As shown in FIGS. 64A and64B, use of contact force and shape data to reach desired points (a),the system can guide the operator along paths with low forces that canlead to other areas of the geometry. As show in FIG. 64B, switching totreatment mode, the system will allow the treatment tip to approachanatomy while controlling the shape to minimize interaction forces(pressures) along the body of the instrument (1).

Mechanical Failure and Structural Integrity of the Instrument:

The use of shape information also allows for determining different typesof mechanical failure. FIG. 65A shows a shapeable instrument (1)assuming a particular configuration. Generally, shapeable instrumentsthat include positioning elements that apply tension but not compressiveforce have two main sources of mechanical failures modes that can bediagnosed with measurement of shape. The first mode includes fracture ofpositioning element that results in a failure to properly position theinstrument (1). The second failure mode includes a fracture in thestructure of the shapeable instrument (1) as noted by area (92). Theability to measure shape of the instrument (1) allows for an immediateindication of a mechanical failure of the instrument (1).

In the case of a fracture in the structure, bending with non-zero forcearound the point of fracture results in higher than normal curvature.The curvature may be higher than expected for a device and lead to animmediate failure diagnosis simply monitoring each point along the pathof the device. An example of a block diagram to assess such a failure isshown in FIG. 65C. The high value shown in (520) can be pre-determinedusing the bending characteristics of the device.

For a more subtle partial fracture, the curvature could be compared toan expected curvature generated by a model and a diagnosis made based onthe cumulative residual (e.g. integral of norm of distance betweenexpected and measured shapes), diagramed in FIG. 65D.

A simple model might be a minimum smoothness of curvature such that adiscontinuity at any point gives rise to indicate a mechanical failure.The expected shape of a device could also be generated by a solidmechanics model taking into account the actuator inputs. The expectedcurvature for each point would be compared to the measurement ofcurvature at that point, iterating along the path of the device.

In the case of fracture of actuating tendons, the curvature would beless than expected for a given level of actuation. If the flexiblesection defaults to a zero-force position, the curvature must becompared to a modeled curvature. The flowchart in FIG. 65D describesdifferentiating between a case of a structural fracture with a highcurvature and a positioning element fracture that give rise to a lowcurvature.

As discussed above, contact with the environment will alter the shapefrom that expected when the instrument is actuated in free space. Suchenvironmental factors can be considered when comparing measuredcurvature to expected curvature to prevent a false indication offailure. The diagnostic algorithm can simply ignore shape measurementswhen in the shapeable instrument (1) contacts an obstruction. If distalforce is measured, it can be included in models that estimate shape.

Dynamics can be used to assess the instrument for failure modes while itis in contact with the environment, meaning when there is an errorbetween the desired shape and measured shapes. To detect a quickmechanical fracture (such as a break or tear), the measured curvature ateach point along the device could be compared with a low-pas filteredversion of itself. If the two differ greatly, a fracture may haveoccurred. The bandwidth of the filter would have to be faster than thebandwidth of actuation.

Following, the actuator inputs could be considered and applied to acurrent estimate of catheter shape. The shape estimate would filter themeasured shape to accommodate contact deflection and apply the actuationrelatively to the estimate. A beneficial by-product of this shapeestimate would be an estimate of when the device makes contact with itsenvironment.

Display of Fracture Information

When a fracture is detected, it is important to convey the failure tothe automated system and user. When the control system is notified of amechanical fracture, tension on tendons should be configured to freezethe actuator state or reduce passive applied force which would likely bethe relaxed tendon positions. In an additional variation, the userinterface could provide haptic feedback to the operator by changing theforce characteristics on the master device (e.g. turn offforce-feedback, centering, gravity compensation, etc.). Also, the systemcould display a visual indicator of device failure. Visual notificationcould be simply textual. Alternatively, the visual notification can beinformation rich. If the system has a visual depiction of theinstrument, the depiction of the instrument could be altered in shape,color, and/or other representative features to reflect the failure.Thus, if the instrument structure fractures, then the operator interfacedisplay could show a displacement of material at the suspected point offracture on the corresponding depiction. If the actual fracture mode isnot visually compelling to demonstrate the degree of correspondingmechanical problem that will result from the failure, the main displaycould show an icon near the area, magnify the area and show the fractureor highlight the area with contrasting colors. Simulating a coloredsemi-transparent light illuminating the fractured section would imitateother alarms such as police revolving lights to give operators anintuitive reaction.

Communicating the point of failure and type if possible is important inthose cases where stopping treatment or operation of the device is notthe chosen fail-safe response to a failure. If the instrument remainsfunctional (albeit impaired), then the system could permit a user withthe option of continuing in a reduced workspace or reducedfunctionality. In such a case, showing the old and new workspace as wellas the fracture point will aid an operator in safely performing theirtasks or exiting the workspace. FIGS. 66A and 66B illustrate twoexamples of visual indication of a failure mode. For example, FIG. 66Ashows a condition where a positioning element fails causing lowcurvature. As shown, the real shape of the device is shown with aphantom desired shape. Also, the fracture is represented by a failureindicator. FIG. 66B illustrates another example where the device remainsoperable but the fracture (92) is indicated by a visual fractureindicator (96). Clearly, any number of additional variations are withinthe scope of this disclosure.

Active Secondary Diagnostic

Because of variation in devices and operating conditions, there is oftensome overlap between acceptable and unacceptable failures in a givendiagnostic metric. Thus, there may be what are statistically called TypeI or Type II errors for the hypothesis that there is a failure (falsefailures/positive or false passes/negative, respectively). To avoidfalse passes in critical diagnostics, the pass-fail criteria may bebiased to report failures when the behavior is questionable due tooperating condition. As one example, if a positioning element fractureswhile a default-straight device is nearly straight it will be difficultto diagnose with the small change in movement from nearly straight tostraight. The system may request an operator perform a specific maneuveror input in order to verify the failure behavior. In the positioningelement fractures situation, the system will request the operator movein the direction the positioning element in question would normallypull. A lack of motion with such intentional motion would clearlyindicate a fracture which could be reported with confidence to the user.

Structural Integrity of the Instrument:

One aspect of using any flexing material is that the bulk properties offlexible sections change over time as the section is repeatedly flexed.Materials may fatigue, slightly surpass their elastic regions or in thecase of composites experience slip on the micro-scale between materials.In extreme cases, these behaviors qualify as a fracture of the material.However, in many cases, less significant changes allow continued use ofthe flexible section but alter the flexural properties of the flexingmaterial and therefore of the shapeable device. If the degradation isfairly continuous and smoothly progressing in time, knowledge of thelevel of degradation could be useful to the system operator. This “stateof the health” of the flexible section as well as other adaptedproperties are useful information for users and extra-controlsubsystems.

The state of health could be conveyed to the user as numericalpercentage of original life or with an icon such as a battery level typeindicator. The state of health can be calculated by comparing measuredbehavior to behavior predicted from a model of the device populated withparameters from the initial manufactured properties. It could also becalculated by comparing the behavior to models populated with parametersof known fatigued devices. This second technique would also provide anestimate of the device properties which may be used to in place of theinitial parameters improve control of the device. Finally, the deviceproperties may be adapted directly by minimizing the residual by varyingthe value of the target property.

Sensor Integrity

In order to diagnose failures of a flexible robotic mechanism, theintegrity of the shape sensor must be known. Differentiating between afailure of the sensor and a failure of that which is sensed can becomplicated depending on the type of sensor used for the measurement.

The first diagnostic of sensor integrity can be based on the physics ofthe sensor itself. First, the range of reasonable outputs of the sensorshould be bounded to enable detection of disconnected wiring. Forexample, if curvature at point x along the flexible path S is measuredas an electrical potential between 1 and 4 Volts, the electrical systemshould be capable of measuring between 0 and 5 Volts. Thus, ameasurement of 0.5 Volts would lead to obvious diagnosis of a sensorfault. The physics of the sensor and measuring system can be consideredin designing this level of diagnostic. When the sensor is measuringoutside the realm of physically reasonable, information about the robotcannot be known.

If more subtle failures of the sensor are possible, a scalar error forexample, lack of sensor integrity could easily be confused for amechanical failure. If the sensor registers an extremely large curvatureat one point, that would seem to indicate a fractured structure. If sucherrors are possible in the sensor though, other information may beconsidered to differentiate sensor integrity from mechanical integrity.This is important in order to properly inform the user what componenthas failed for repair or replacement and for the control system toimplement a fail-safe response.

In this case, model based diagnostics may be used to combine othersources of information to decide which component has failed.

If tip position is measured, a sudden movement of the tip position couldindicate a real fracture. If the shape sensor measures a change thatshould move the tip position, but no tip motion is detected, the sensormay be faulted.

The current shape and tip position of the device can be predicted basedon a model of the device, the previous shape measurement and theprevious tip position measurement. Thus, the measured shape and tipposition may be compared to their predicted values to produce residuals.These residuals may be the output of a Kalman filter or other suchestimator. The residuals may then be analyzed to decide which failure toreport. They may each simply be compared to a threshold, the residualmay be integrated then compared to a threshold, or the residuals may beincorporated with operating conditions in a Bayesian network trainedwith operating and failed components.

Tip orientation can be used analogously or combined with to tipposition.

If a measured shape has been registered to a hollow geometricenvironment model, the device should pass through the model surfaceboundaries. If the sensor measurement leaves the model, the sensor isprobably erroneous.

If the measured device shape changes rapidly but there is no change inactuating wire-tensions, the device probably did not fracture and thusthe sensor is probably erroneous.

Similarly, if the device is held in a state with natural potentialenergy by actuators connected with a back drivable drive train and themeasured shape changes but the actuator effort does not, the deviceprobably did not fracture and thus the sensor is probably erroneous.

The systems described herein can predict the current shape and otherproperties the device and actuators based on a model of the device, theprevious shape measurement and other estimates of previous properties.Thus, comparing measured shape and other measurements to their predictedvalues provides residuals. These residuals may be the output of a Kalmanfilter or other such estimator. The residuals may then be analyzed todecide which failure to report. They may each simply be compared to athreshold, the residual may be dynamically transformed then compared toa threshold, or the residuals may be incorporated with operatingconditions in a Bayesian network trained with operating and failedcomponents.

While multiple embodiments and variations of the many aspects of theinvention have been disclosed and described herein, such disclosure isprovided for purposes of illustration only. Many combinations andpermutations of the disclosed system are useful in minimally invasivemedical intervention and diagnosis, and the system is configured to beflexible. The foregoing illustrated and described embodiments of theinvention are susceptible to various modifications and alternativeforms, and it should be understood that the invention generally, as wellas the specific embodiments described herein, are not limited to theparticular forms or methods disclosed, but also cover all modifications,equivalents and alternatives falling within the scope of the appendedclaims. Further, the various features and aspects of the illustratedembodiments may be incorporated into other embodiments, even if no sodescribed herein, as will be apparent to those skilled in the art.

We claim:
 1. A method of controlling advancement of a shapeable medicaldevice within an anatomic path, the method comprising: processing acommand from an input device to actuate the shapeable medical deviceinto a desired configuration of one or more portions of the shapeablemedical device; issuing a position control signal to one or moreactuators based on a model of shapeable medical device mechanics toadvance the shapeable medical device along the anatomic path; obtaininga plurality of localization data to determine a real shape of at leastthe one or more portions of the shapeable instrument when advanced alongthe anatomic path; and determining a differential between the real shapeand the desired configuration of the one or more portions; and based onthe differential between the real shape and the desired configuration ofthe one or more portions, updating the model of shapeable medical devicemechanics and controlling advancement of the shapeable medical devicebased on the updated model.
 2. The method of claim 1, where controllingthe advancement of the shapeable medical device comprises reversing theshapeable medical device along the anatomic path until the differentialbetween the real shape and the desired configuration decreases.
 3. Themethod of claim 1, where controlling the advancement of the shapeablemedical device comprises slowing advancement of the shapeable medicaldevice along the anatomic path until the differential between the realshape and the desired configuration decreases.
 4. The method of claim 1,where controlling the advancement of the shapeable medical devicecomprises advancing a guide device from the shapeable medical devicewithin the anatomic path and subsequently advancing the shapeablemedical device along the guide track.
 5. The method of claim 1, wherecontrolling the advancement of the shapeable medical device comprisesstopping the shapeable medical device and withdrawing a proximal end ofthe shapeable medical device until the differential between the realshape and the desired configuration decreases.
 6. The method of claim 1,wherein the model of shapeable medical device mechanics is based in parton a historical database of real shapes of the one or more portions ofthe shapeable medical device.
 7. The method of claim 1, wherein thedesired configuration is determined based on commanded catheterconfigurations.
 8. The method of claim 1, wherein the desiredconfiguration is indicative of an anatomical path.